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Executive  Summary 


A.  Overview 

This  task  was  assigned  to  the  Institute  for  Defense  Analyses  (IDA)  by  the  Direc¬ 
tor,  Biosystems,  Office  of  the  Deputy  Under  Secretary  of  Defense  (Science  and  Technol¬ 
ogy)  (ODUSD(S&T)).  The  purpose  of  this  task  was  to  assess  contributions  made  by  the 
Department  of  Defense  (DoD)  Small  Business  Innovation  Research  (SBIR)  and  Small 
Business  Technology  Transfer  (STTR)  programs  to  science  and  technology  (S&T)  in  the 
area  of  training  and  education. 

This  document  distinguishes  between  SBIR/STTR  “topics”  and  “projects.”  In  this 
context,  solicitation  topics  are  developed  by  government  scientists  and  program 
managers  who  then  act  as  technical  managers  for  projects  funded  under  the  topics. 

SBIR/STTR  project  work  is  divided  into  three  phases: 

•  Phase  1  supports  exploration  of  the  technical  merits  of  an  idea  or  technology. 

•  Phase  2  supports  production  of  a  research  and  development  (R&D)  product 
and  evaluation  of  its  commercial  potential. 

•  Phase  3  supports  commercialization  of  the  product — its  entry  into  the 
marketplace. 

This  review  included  80  SBIR/STTR  topics.  The  criteria  for  selecting  topics  were 
that  they  should  focus  on  training  and  education,  target  a  credible  innovation,  begin 
under  FY  1999  through  FY  2004  funding,  and  make  at  least  one  Phase  2  award. 

FY  1999  was  selected  as  the  beginning  year  for  topics  to  include  in  this  effort 
because  of  the  review  of  training  and  education  R&D  performed  in  that  year.  The  1999 
workshop  was  not  limited  to  SBIR/STTR  topics  and  was  organized  in  a  top-down  manner 
that  focused  on  the  following  R&D  topic  areas  in  training  and  education: 

•  Intelligent  Tutoring  Systems  (ITSs) 

•  Authoring  Tools 
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•  Distributed  Simulation  Environments  for  Instruction 

•  Dynamic  Learning  Management. 

The  FY  1999-FY  2004  SBIR/STTR  topics  have  been  categorized  in  much  the 
same  way  as  the  efforts  reviewed  in  1999.  However,  the  SBIR/STTR  topics  included  in 
this  review  generally  reflect  the  applied,  entrepreneurial,  and  commercial  aims  of  the 
SBIR/STTR  program  rather  than  a  concentration  on  S&T  progress.  They  also  reflect  the 
specific  criteria  used  to  select  topics  for  this  SBIR/STTR  review. 

Four  categories  of  R&D  efforts  were  identified  for  the  FY  1999-FY  2004  review: 

1 .  Intelligent  Tutoring  Systems  (ITSs) 

2.  Authoring  Tools 

3.  Simulation-Based  Training 

4.  Training  System  Design  and  Development. 

B.  DoD  SBIR/STTR  for  Training  and  Education 
1.  Investment 

DoD  investment  in  SBIR/STTR  training  and  education  during  FY  1999-FY  2004 
was  more  than  $83  million,  which  was  awarded  to  282  projects  (184  Phase  1  efforts  and 
98  Phase  2  efforts).  This  investment  grew  substantially  over  these  years,  rising  from 
$0.9  million  in  FY  1999  to  $24.6  million  in  FY  2003.  The  investment  was  lower  in 
FY  2004  ($19.4  million),  but  it  is  still  greater  by  a  factor  of  about  20  over  FY  1999. 

More  specifically,  over  the  FY  1999-FY  2004  time  period: 

•  The  Air  Force  sponsored  23  topics  ($22.8  million  awarded  for  64  Phase  1 
and  Phase  2  projects),  the  Army  sponsored  20  topics  ($16.8  million  awarded 
for  49  Phase  1  and  Phase  2  projects),  and  the  Navy  sponsored  5  topics 
($3.8  million  awarded  for  17  Phase  1  and  Phase  2  projects). 

•  ODUSD(S&T),  through  the  Director,  Biosystems,  sponsored  31  topics  and 
awarded  $35.7  million  for  140  Phase  1  and  Phase  2  projects.  Its  topics  were 
defined,  nominated,  and,  when  selected,  managed  by  the  Services  and  the 
Defense  Advanced  Research  Projects  Agency  (DARPA).  All  three  Services 
(Army,  Navy,  Air  Force)  participated  about  equally  in  this  activity. 

•  SBIR/STTR  investments  in  intelligent  tutoring  supported  1 1  topics,  with 
$15  million  awarded  for  45  Phase  1  and  Phase  2  projects.  Authoring  Tools 
(21  topics,  $22  million,  and  81  projects),  Simulation-Based  Training 
(26  topics,  $23  million,  and  76  projects),  and  Training  System  Design  and 
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Development  (22  topics,  $23  million,  and  80  projects)  were  roughly  equiva¬ 
lent  in  the  number  of  topics,  the  funds  awarded,  and  the  number  of  projects. 
About  42  of  the  80  SBIR/STTR  topics  chosen  for  this  review  focused  on 
developing  training  packages  for  specific  subject  areas. 

•  The  Army  led  in  proportion  of  Phase  2  to  Phase  1  awards,  with  69  percent  of 
Phase  1  awards  progressing  to  Phase  2.  The  Air  Force  was  fairly  close  with 
60  percent.  The  Office  of  the  Secretary  of  Defense  (OSD)  and  the  Navy 
Phase  1  projects  were  less  likely  to  progress  to  Phase  2  (46  and  42  percent, 
respectively). 

2.  Goals 

R&D  goals  for  topics  and  projects  in  this  review  were  identified  as  the  following: 

•  ITSs 

-  Natural  language  tutorial  dialogue 

-  Automated,  continuous,  unobtrusive  assessment  of  learner  state  and 
progress 

-  Representation  of  knowledge,  skill,  and  ability 

-  Dynamic  assembly  of  training  from  instructional  objects  [dynamic 
assembly  of  instructional  interventions  that  help  the  learner  attain  tar¬ 
geted  instructional  objectives  (i.e.,  help  the  learner  progress  from  the 
current  state  of  knowledge,  skill,  and  ability  to  the  objective  state]. 

•  Authoring  Tools 

-  Linking  design  and  development  to  instructional  objectives 

-  Authoring  by  non-information  Technology  (IT)  specialists 

-  Simulation  and  scenario  generation  and  preparation 

-  Rapid  (agile)  simulation  and  scenario  authoring 

-  Creation  of  computer-generated  (automated  and  semi-automated) 
participants  (allies,  enemies,  and  neutrals). 

•  Simulation-Based  Training 

-  Simulation  training  linked  to  instructional  objectives  (for  individuals) 

-  Simulation  training  linked  to  instructional  objectives  (for  teams) 

-  Post-simulation  performance  feedback 

-  Synthetic  agents  and  avatars  in  simulation 

-  Haptics  in  medical  training. 
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•  Training  System  Design  and  Development 

-  Team  training 

-  Cognitive  readiness  training  for  leaders 

-  Training  in  areas  of  special  concern. 

•  Overall  (goals  addressed  across  the  four  R&D  categories) 

-  Maintain  focus  on  military  relevance 

-  Increase  accessibility  of  training,  education,  and  performance  aiding 

-  Incorporate  “intelligent”  computational  capabilities 

-  Increase  agility,  rapidity,  and  ease  of  use  in  development 

-  Provide  for  reuse  and  interoperability 

-  Develop  instructional  interventions  related  to  training  and  education 
objectives 

-  Develop  simulation  and  games  for  training  and  education 

-  Tailor  training  and  education  to  learner  capabilities  and  needs. 

3.  Gaps 

Some  identified  S&T  gaps  for  the  four  R&D  categories  appear  aligned  with  the 
R&D  goals  just  listed. 

•  ITSs.  Natural  language  dialogue;  learner  and  expertise  modeling;  and  gen¬ 
erating  tailored  instructional  interactions. 

•  Authoring  tools.  Object-oriented  authoring  tools;  authoring  tools  for  non-IT 
personnel;  cost-effectiveness  tradeoffs;  and  authoring  tools  for  specific 
capabilities. 

•  Simulation-based  training.  Matching  simulation-based  training  to  training 
objectives;  automated  detection  of  critical  events  and  decisions;  cost-effec¬ 
tive  application  of  simulation-  and  game-based  training;  and  development  of 
specific  simulation  capabilities. 

•  Training  systems  design  and  development.  Team  training;  cognitive  readi¬ 
ness  training  and  assessment;  and  specific  training  techniques  (e.g.,  language 
and  cultural  training). 

•  Overall.  Gaps  that  appeared  to  be  suitable  for  SBIR/STTR  support  across  all 
four  areas  included  development  of  practicable  cost-effectiveness  models; 
progress  assessment  from  routine  interactions;  generation  of  instructional 
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materials  from  reusable  objects;  and  instruction  techniques  for  enhancing 
cognitive  readiness  (agility,  flexibility,  creativity). 


4.  Management 

Observations  on  SBIR/STTR  management  for  training  and  education  include 

•  Selecting  SBIR/STTR  topics  for  training  and  education.  Management 
attention  and  leadership  may  be  needed  to  ensure  that  human  systems  exper¬ 
tise  is  included  in  final  selection  of  SBIR/STTR  topics  and  that  well-defined 
benchmarks  are  applied  in  assigning  scores  during  the  selection  of  proposals 
for  funding. 

•  Ensuring  innovation.  Topics  and  projects  selected  for  SBIR/STTR  funding 
should  be  those  that  pay  serious  attention  to  innovation  and  product 
production. 

•  Infrastructure  engineering.  Attention  and  resources  should  be  allocated  for 
the  development  of  an  infrastructure  engineering  capability  and  discipline  to 
ensure  that  returns  on  SBIR/STTR  investments  are  fully  and  successfully 
realized. 

•  Oversight  for  commercialization.  Promising  SBIR/STTR  innovations  and 
products  are  too  often  abandoned  in  the  laboratory.  Their  implementation  and 
commercialization  might  be  improved  if  more  government  oversight  were 
exercised  to  ensure  that  R&D  firms  entering  Phase  2  incorporate  and 
embrace  the  culture  of  commercialization. 

•  Evaluating  project  results.  More  attention  should  be  paid  to  evaluating  the 
products  of  SBIR/STTR  projects  and  documenting  their  S&T  value. 

•  Submission  of  project  reports.  DoD  requires  the  submission  of  SBIR/STTR 
reports  to  the  Defense  Technical  Information  Center  (DTIC);  however,  this 
requirement  is  not  always  met.  SBIR/STTR  projects  develop  innovative 
products,  procedures,  and  technologies  that  can  contribute  significantly  to 
training  and  education  S&T.  Project  results  should  be  reported  and  made 
available. 
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Contributions  of  the  Department  of  Defense  Small  Business  Innovation 
Research  (SBIR)  and  Small  Business  Technology  Transfer  (STTR) 
Programs  to  Training  and  Education:  FY  1999-FY  2004 


A.  Purpose 

The  Director,  Biosystems,  Office  of  the  Deputy  Under  Secretary  of  Defense  (Sci¬ 
ence  and  Technology)  (OUSD(S&T))  assigned  this  task  to  the  Institute  for  Defense 
Analyses  (IDA).  The  purpose  of  this  task  was  to  assess  contributions  made  by  Depart¬ 
ment  of  Defense  (DoD)  Small  Business  Innovation  Research  (SBIR)  and  Small  Business 
Technology  Transfer  (STTR)  programs  to  science  and  technology  (S&T)  in  the  area  of 
training  and  education.  More  specifically,  IDA  was  tasked  to 

•  Review  topics1  that  concerned  training  and  education  and  were  addressed  in 
the  DoD  SBIR/STTR  program  during  the  FY  1999-FY  2004  time  period 

•  Assess  the  S&T  products  and  progress  of  DoD  SBIR/STTR  investments  in 
training  and  education  during  the  FY  1999-FY  2004  time  period 

•  Detennine  how  the  SBIR/STTR  topics  might  be  aggregated  and  described 
overall  as  a  research  and  development  (R&D)  program 

•  Identify  significant  S&T  gaps  that  need  to  be  filled  to  achieve  the  objectives 
of  this  R&D  program. 

B.  SBIR/STTR  Background 

The  government’s  SBIR  program  arose  from  the  recognition  in  the  1970s  that 
small  businesses  had  become  a  significant  economic  factor  in  creating  jobs  and  pio¬ 
neering  technological  innovation  (National  Research  Council,  1999).  However,  at  that 
time,  the  share  of  federal  R&D  funds  being  awarded  small  businesses  appeared  to  be 
declining. 


1  This  document  distinguishes  between  SBIR/STTR  “topics”  and  “projects.”  In  this  context,  solicitation 
topics  are  developed  by  government  scientists  and  program  managers,  who  then  may  act  as  technical 
managers  for  projects  funded  under  the  topics. 
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In  January  1980,  the  Administration  convened  a  White  House  Conference  on 
Small  Business  to  determine  if  support  for  small  business  research  and  innovation  should 
be  expanded.  The  Conference  found  that  small  businesses  were  generating  two  and  a  half 
times  as  many  innovations  per  employee  as  large  businesses  but  that  large  businesses 
were  receiving  three  times  more  government  assistance.  The  Conference  therefore  rec¬ 
ommended  that  legislation  be  enacted  to  expand  R&D  support  for  small  businesses.  This 
recommendation  resulted  in  the  Small  Business  Innovation  Development  Act  of  1982. 
This  Act,  whose  purpose  was  to  strengthen  the  role  of  small  businesses  in  federally 
funded  R&D  and  help  develop  a  stronger  national  base  for  technical  innovation,  had  four 
specific  objectives  (National  Research  Council,  1999): 

1 .  Stimulate  technological  innovation 

2.  Increase  the  use  of  small  businesses  to  meet  federal  R&D  needs 

3.  Foster  and  encourage  participation  by  minority  and  disadvantaged  persons  in 
technological  innovation 

4.  Increase  private  sector  commercialization  of  innovations  derived  from 
federal  R&D. 

The  SBIR  program  was  reauthorized  in  1992  and  included  the  addition  of  the 
STTR  program.  The  objectives  of  the  STTR  program  are  effectively  the  same  as  those  of 
the  SBIR  program  but  include  the  addition  of  nonprofit  research  institutions  such  as  uni¬ 
versities  and  Federally  Funded  Research  and  Development  Centers  (FFRDCs).  The  main 
practical  difference  between  the  SBIR  and  STTR  programs  (aside  from  the  much  greater 
size  of  the  SBIR  program)  is  that  STTR  projects  require  cooperative  and  collaborative 
research  between  a  small  business  firm  (which  assumes  primary  responsibility  for  the 
project)  and  a  nonprofit  research  institution. 

In  SBIR  and  STTR  programs,  a  small  business  is  defined  as  a  for-profit  firm 
employing  500  or  fewer  employees.  In  practice,  over  80  percent  of  SBIR/STTR  funds  are 
awarded  to  firms  that  have  less  than  100  employees.  No  size  limit  is  specified  for  non¬ 
profit  research  entities  in  STTR  awards,  but  the  small  business  firm  to  which  the  award  is 
made  must  manage  and  control  the  STTR  funding.  At  least  40  percent  of  the  work  must 
be  perfonned  by  the  business  firm,  and  at  least  30  percent  of  the  work  must  be  performed 
by  the  research  institution.  All  SBIR/STTR  awards  must  go  to  firms  that  are  at  least 
51  percent  owned  and/or  controlled  by  United  States’  citizens.  The  work  funded  must  be 
performed  in  the  United  States. 
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The  1982  bill  was  funded  by  a  “tax”  of  0.2  percent  on  the  R&D  budget  of  all  fed¬ 
eral  agencies  whose  R&D  budgets  exceeded  $100  million.  Today,  the  SBIR/STTR  legis¬ 
lation  requires  federal  agencies  that  have  extramural2  research  expenditures  of  more  than 
$100  million  to  set  aside  2.5  percent  of  their  R&D  budgets3  for  SBIR  and  0.3  percent  for 
STTR  (National  Research  Council,  2004).  Awards  for  all  phases  of  SBIR/STTR  work  in 
FY  2004  exceeded  $2  billion.  Slightly  more  than  50  percent  of  the  SBIR/STTR  funding 
came  from  DoD,  which  supported  1,082  SBIR  topics  and  118  STTR  topics  in  FY  2004. 
The  SBIR/STTR  programs  were  reauthorized  in  2000  (Public  Law  106-554)  and  are  now 
authorized  to  continue  through  FY  2008. 

SBIR/STTR  project  work  is  divided  into  three  phases: 

•  Phase  1.  Phase  1  provides  awards  up  to  $100,000  of  effort  for  about 
6  months  to  support  the  exploration  of  the  technical  merits  of  an  idea  or  a 
technology.  Work  proceeds  to  Phase  2  based  on  Phase  1  perfonnance  and 
results. 

•  Phase  2,  Phase  2  provides  awards  up  to  $750,000  for  about  2  years  to  pro¬ 
duce  the  actual  R&D  product  and  evaluate  its  commercial  potential.  No 
SBIR/STTR  funds  are  provided  for  Phase  3,  which  is  intended  to  transition 
the  Phase  2  innovation  into  the  marketplace. 

•  Phase  3.  Phase  3  funds  must  come  from  the  private  sector  or  from  another 
non-SBIR/STTR  source.  However,  some  agencies,  including  DoD,  provide 
“extended  funding”  for  Phase  2.  These  funds  can  support  further  develop¬ 
ment  of  innovations  as  products  for  the  specific  use  by  the  agency  that  pro¬ 
vides  them. 

C.  SBIR/STTR  Topic  Selection 

The  criteria  for  including  SBIR/STTR  topics  in  this  review  were  that  they  must 

•  Be  focused  primarily  on  training  and  education.  This  first  criterion  was 
adopted  to  ensure  the  inclusion  of  topics  intended  to  develop  new  capabilities 
for  the  teaching-learning  processes.  This  criterion  resulted  in  the  exclusion  of 
topics  that  primarily  emphasized  the  following: 

-  Support  issues,  such  as  personnel  management  or  the  engineering  design 
of  simulators  or  simulations  rather  than  direct  enhancement  of  learning 


2  Activities  not  performed  by  agency  employees. 

3  For  DoD,  the  Research,  Development,  Test,  &  Evaluation  “Program  6”  budget. 


3 


-  Short-term  performance  aiding  (e.g.,  problem  solving,  decision  aiding, 
troubleshooting)  rather  than  the  longer  term  development  of  knowledge, 
skill,  and  ability  (KSA)4  targeted  by  training  and  education. 

•  Target  a  credible  innovation  and/or  S&T  advance.  This  second  criterion 
was  adopted  to  ensure  that  the  selected  topics  focused  attention  on 
innovation  and  advancing  S&T  rather  than  on  an  exclusive  concern  with 
producing  a  training  or  education  program. 

•  Have  been  initiated  under  FY  1999  through  FY  2004  funding.  This  third 
criterion  was  adopted  to  determine  what  S&T  progress  in  training  and  educa¬ 
tion  had  been  contributed  by  the  DoD  SBIR/STTR  program  in  the  5  years 
since  ODUSD  (S&T)  had  conducted  an  earlier  (1999)  comprehensive  review 
of  training  and  education  R&D. 

•  Have  awarded  at  least  one  Phase  2  project.  This  fourth  criterion  was 
adopted  to  ensure  that  the  topic  either  had  awarded  an  identifiable  product  or 
was  about  to  do  so.  Most,  but  not  all,  SBIR/STTR  topics  award  at  least  one 
Phase  2  project. 

Many  topics  included  in  this  review  were  identified  using  the  DoD  SBIR/STTR 
database  provided  at  http://www.dodsbir.net/.  Candidate  topics  were  initially  identified 
by  searching  for  projects  that  were  initiated  in  the  FY  1999-FY  2004  time  frame  and 
included  the  words  “training”  or  “education”  in  their  title,  keywords,  and/or  abstract. 
Additional  candidate  topics  were  identified  through  personal  contacts  with  government 
project  and  program  managers.  Descriptions  and  objectives  of  candidate  SBIR/STTR 
topics,  along  with  reports  of  the  projects  they  awarded,  were  then  reviewed  in  accord 
with  the  four  criteria  listed  above. 

D.  The  June  2005  SBIR/STTR  Workshop 

A  large  and  productive  SBIR/STTR  workshop  was  hosted  by  the  Human  Systems 
Reliance  Panel,  the  U.S.  Army  Research  Institute  for  the  Behavioral  and  Social  Sciences 
(ARI),  and  the  United  States  Joint  Forces  Command  (USJFCOM)  on  20-24  June  2005. 
This  review  includes  67  of  the  SBIR/STTR  topics  discussed  during  the  workshop.  Based 
on  reports  and  information  acquired  through  the  end  of  August  2005,  13  topics  were 
added  following  the  workshop,  yielding  the  total  of  80  SBIR/STTR  topics  included  in 


4  Training  objectives  are  often  expressed  as  knowledge,  skills,  and  abilities  (KSAs  in  training  parlance), 
where  “ability”  generally  refers  to  the  correct  application  of  the  knowledge  and  skills  attained.  The 
“A”  in  KSAs  just  as  often  refers  to  attitude,  but  that  alternative  is  not  addressed  in  this  report. 
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this  review.  Appendix  A  lists  these  topics  and  includes  a  brief  description  of  their 
objectives. 

The  purposes  of  the  June  2005  SBIR/STTR  Workshop  included  those  listed  for 
this  task:  assess  the  contributions  of  SBIR/STTR  activity  to  training  and  education  S&T; 
identify  clusters  of  topics  that  could  be  aggregated  to  describe  and  characterize  DoD 
SBIR/STTR  activity  in  training  and  education  overall;  identify  S&T  products;  and  iden¬ 
tify  S&T  gaps.  The  workshop  was  also  intended  to  improve  communication  and  coordi¬ 
nation  among  government  R&D  managers,  thereby  reinforcing  and  improving  the 
development  of  the  DoD  community  of  interest  in  training  and  education. 

1.  SBIR/STTR  Topic  Organization 

The  June  2005  Workshop  brought  together  a  host  of  government  participants, 
topics,  and  activities.  Its  concern  with  training  and  education  echoed  the  earlier  1999 
workshop  that  the  OUSD(S&T)  Director  of  Biosystems  had  organized  to  review  training 
and  education  R&D  in  general. 

The  1999  workshop  was  not  limited  to  SBIR/STTR  topics  and  was  organized  in  a 
top-down  manner  that  focused  on  the  following  R&D  topic  areas  in  training  and 
education: 

•  Intelligent  Tutoring  Systems  (ITSs) 

•  Authoring  Tools 

•  Distributed  Simulation  Environments  for  Instruction 

•  Dynamic  Learning  Management. 

Aggregates  of  SBIR/STTR  topics  for  the  2005  review  and  workshop  were  identi¬ 
fied  in  a  bottom-up  manner  to  allow  the  SBIR/STTR  topics  themselves  detennine  areas 
of  R&D  activity.  Even  though  the  topics  reflect  the  applied,  entrepreneurial,  and  com¬ 
mercial  mandate  of  the  SBIR/STTR  programs,  the  R&D  areas  identified  in  2005  closely 
resembled  those  identified  for  the  1999  workshop: 

•  ITSs 

•  Authoring  Tools 

-  General  authoring  tools 

-  Authoring  tools  for  simulation 

-  Authoring  tools  for  computer-generated  participants 
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•  Simulation-Based  Training 

-  Simulation  training  for  teams 

-  Simulation  training  for  individuals 

•  Training  System  Design  and  Development. 

These  areas  are  used  in  this  review  to  describe  and  characterize  DoD  SBIR/STTR 
activity  in  education  and  training.  SBIR/STTR  investment  has  produced  S&T  advances 
in  each  of  them.  Other  areas  might  reasonably  have  been  used,  but  those  listed  above 
appeared  to  provide  the  best  and  most  comprehensive  fit  for  SBIR/STTR  topics  created 
and  funded  in  FY  1999-FY  2004. 

Table  1  shows  the  SBIR/STTR  R&D  areas  and  the  number  of  topics  presented  in 
each  area  at  the  June  2005  Workshop  by  the  Army,  the  Navy,  the  Air  Force,  the  Office  of 
the  Secretary  of  Defense  (OSD),  and  the  Defense  Advanced  Research  Projects  Agency 
(DARPA). 


Table  1.  Number  of  Topics  by  Service/Organization  and  by  Sub-Areas 


R&D  Areas 

Arm 

y 

Navy 

Air  Force 

OSD 

DARPA 

Totals 

ITSs 

4 

- 

1 

5 

1 

11 

Authoring  Tools 

General  authoring  tools 

1 

1 

1 

7 

- 

10 

Authoring  tools  for 
simulations 

1 

1 

4 

3 

- 

9 

Authoring  tools  for 

computer-generated 

participants 

1 

1 

2 

Simulation-Based  Training 

Simulation  training  for 
teams 

3 

- 

5 

1 

- 

9 

Simulation  training  for 
individuals 

7 

- 

8 

2 

- 

17 

Training  System  Design 
and  Development 

3 

3 

3 

13 

- 

22 

Totals 

20 

5 

23 

31 

1 

80 

Proportion  of  total 

0.25 

0.06 

0.29 

0.39 

0.01 

100.0 
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All  topics  chosen  for  this  review  are  listed  in  accord  with  this  organization  and 
are  briefly  described  in  Appendix  A.5 

a.  Intelligent  Tutoring  Systems  (ITSs) 

Efforts  to  imbue  computer-mediated  environments  with  intelligence  and  the  capa¬ 
bility  to  participate  in  instructional  interactions  (dialogues)  generated  on-demand  and  in 
real  time  have  been  viewed  for  some  time  as  requiring  four  interdependent  capabilities 
(e.g.,  Brown,  Burton,  and  DeKleer,  1982;  Fletcher,  1975;  Foster  and  Fletcher,  2002): 

1 .  The  capability  to  model  the  knowledge,  skill,  and  ability  possessed  by  indi¬ 
vidual  learners 

2.  The  capability  to  model  the  knowledge,  skill,  and  ability  learners  need  to 
meet  instructional  objectives 

3.  The  instructional  techniques  needed  by  learners  to  help  them  acquire  the 
requisite  knowledge,  skill,  and  ability 

4.  A  means  to  communicate  with  learners — typically  using  interactions  that  can 
be  initiated  either  by  the  learner  or  the  computer,  preferably  undertaken  using 
natural  language. 

ITSs  use  computational  inference  capabilities  to  build  and  then  use  (1)  explicit, 
dynamic  models  of  individual  student’s  needs  and  abilities,  (2)  the  knowledge,  skills,  and 
abilities  to  be  learned  by  each  student,  and  (3)  one-on-one  tutorial  techniques  to  help  stu¬ 
dents  advance  from  their  current  state  of  learning  to  the  objective  state.  Computer-medi¬ 
ated  learning  environments  have  used  similar  capabilities  since  the  mid-1950s,  but  their 
modeling  and  inference  techniques  tended  to  be  more  implicit  than  those  found  in  ITSs 
(Fletcher  and  Rockway,  1986).  Two  features  primarily  set  these  systems  apart  from  ear¬ 
lier  systems  and  led  DoD  to  focus  R&D  on  these  systems  in  the  early  1970s: 

1.  Mixed-initiative  tutorial  dialogues  with  students — dialogues  in  which  either 
the  student  or  the  computer  system  could  initiate  tutorial  inquiries  and  dia¬ 
logue  (Carbonell,  1970) 

2.  Generation  of  instructional  interactions  and  interventions  as  needed  on- 
demand  and  in  real  time  (Brown,  Burton,  and  DeKleer,  1982). 


^  Those  who  attended  the  June  workshop  will  note  that  some  topics  have  migrated  to  different  areas. 
Information  presented  at  this  workshop  and  a  more  careful  review  of  topic  reports  suggested  these 
reassignments. 
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In  the  1970s,  the  capabilities  of  computer-mediated  environments  to  reduce  costs 
and  time-to-leam  while  ensuring  that  each  student  achieved  the  required  level(s)  of  com¬ 
petency  were  recognized  and  accepted.  What  hindered  the  adoption  and  implementation 
of  computer-mediated  environments,  in  addition  to  the  usual  problems  of  administrative 
and  cultural  inertia,  were  the  high  cost(s)  of  developing  and  delivering  computer-medi¬ 
ated  instruction.  It  was  assumed  that  the  almost  irresistible  advance  of  electronics  (pow¬ 
ered  by  Moore’s  Law)6  would  sooner  or  later  solve  the  delivery  problem.  What  remained 
were  the  costs  to  develop,  or  “author,”  these  environments.  The  argument  that  motivated 
DoD  investment  in  ITS  at  the  time  was  that  development  costs  could  be  reduced  substan¬ 
tially  by  enabling  the  technology  itself  to  generate  instruction  content  (e.g.,  instructional 
items,  progress  assessment,  simulations,  and  simulation  scenarios)  and  instructional 
interactions  (e.g.,  tutorial  discussion,  hints,  and  critiques)  (Fletcher  and  Rockway,  1986). 

The  aim  of  developing  computer-mediated  environments  that  can  establish  and 
sustain  tutorial  conversations  with  individual  students  or  with  students  acting 
individually  or  collaboratively  in  teams,  crews,  staffs,  and  units  remains  a  major 
scientific  and  technological  objective  (Chipman,  2003;  Fletcher,  2006;  Foster  and 
Fletcher,  2002;  Graesser,  Gernsbacher,  and  Goldman  2003).  Substantial  progress  has 
been  made  since  the  1970s,  but  more  is  needed  before  training,  education,  and 
performance  aiding  (e.g.,  problem  solving,  troubleshooting,  decision  aiding)  become  true 
mixed-initiative  interactions  between  students  (or  groups  of  students)  and  computers. 
This  objective,  with  its  ability  to  deliver  tailored  instruction  and  perfonnance  aiding, 
remains  as  appealing  as  ever  from  both  cost  and  effectiveness  standpoints.  Government 
R&D  managers  have  continued  to  pursue  it. 

Many  SBIR/STTR  topics  mentioned  intelligent  capabilities  in  their  descriptions, 
but  only  those  that  addressed  a  generative  or  inference-making  capability  were  included 
in  this  area.  Those  that  did  not,  but  that  satisfied  the  other  criteria  for  selection,  were 
placed  in  other  areas  for  this  review. 

As  shown  in  Table  1,  11  SBIR/STTR  topics  were  included  in  the  “ITSs”  area 
(10  of  these  topics  were  presented  at  the  June  2005  Workshop,  and  one  was  added  later). 


6  The  original  Moore’s  Law  derives  from  a  speech  given  by  Gordon  Moore,  later  a  founder  of  Intel,  in 
1965,  in  which  he  observed  that  the  number  of  microcomponents  that  could  be  placed  in  an  integrated 
circuit  (microchip)  with  the  lowest  manufacturing  cost  was  doubling  every  year  and  that  this  trend 
would  likely  continue  into  the  future  (Mann,  2000;  Service,  1996). 
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b.  Authoring  Tools 

Authoring  tools  have  been  an  integral  part  of  computer-mediated  instructional 
environments  from  the  beginning.  The  Air  Force  appears  to  have  been  the  first  of  the 
Services  to  use  computer  capabilities  for  instruction.  The  Air  Force  provided  a  system 
imaginatively  named  the  System  Training  Program  (STP)  to  support  the  Semi-Automated 
Ground  Environment  (SAGE).  SAGE  grew  out  of  the  1950s  Whirlwind  I  project  to  pro¬ 
vide  a  Defense  Early  Warning  system  (the  “DEW  line”)  to  defend  against  air  attacks  on 
the  continental  United  States.  SAGE  included  authoring  tools  that  allowed  training  spe¬ 
cialists  to  program  its  training  exercises  (Fletcher  and  Rockway,  1986). 

Enabling  administrators,  instructors,  and  students  who  are  not  computer  special¬ 
ists  to  create  and  modify  computer-mediated  instructional  environments  to  meet  their 
specific  instructional  needs  and  objectives  has  also  been  a  goal  since  the  beginning.  The 
first  computer  system  specifically  designed  and  used  for  instruction  (the  1965  IBM  1500 
System)  included  Coursewriter,  an  authoring  language  that  was  intended  to  be  used  by 
classroom  teachers  to  prepare  material  for  their  students — even  though  that  was  quickly 
shown  to  be  impracticable.  The  early  Naval  Tactical  Data  System  (NTDS)  included  Les¬ 
son  Translator  (L-TRAN)  as  an  authoring  tool.  L-TRAN  was  used  by  sailors  to  prepare 
training  materials  on  the  operation  of  NTDS.  There  are  now  numerous  additional  exam¬ 
ples  of  the  successful  application  of  authoring  tools.  A  recent  review  of  commercially 
available  and  successfully  used  authoring  tools  covered  43  such  systems  (Nantel,  2004). 

Still,  the  development  (authoring)  of  computer-mediated  instructional  environ¬ 
ments — generative  or  otherwise — remains  an  issue  in  reducing  development  costs  and 
maintaining,  if  not  increasing,  the  quality  of  these  environments  (Gibbons  and  Fair- 
weather,  1998;  Murray,  Blessing,  and  Ainsworth,  2003).  Allowing  professional  develop¬ 
ers,  subject  matter  experts  (SMEs),  and  front-line  instructors  who  are  not  Information 
Technology  (IT)  specialists  to  create  high-quality  learning  environments  in  a  flexible  and 
cost-effective  manner  remains  a  primary  concern. 

Many  SBIR/STTR  authoring  topics  focused  on  this  goal.  These  topics 
emphasized  the  importance  of  enabling  commanders,  instructors,  and  students  to  tailor 
instructional  environments  for  their  immediate  needs  and  audiences.  Imbuing  technology 
with  generative  capabilities  should  increase  the  range,  scope,  and  sheer  amount  of 
instruction  produced  by  a  single  authoring  effort,  but  it  will  not  remove  the  need  for 
human  intervention  in  the  authoring  process. 
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Subtopics  for  the  authoring  tools  area  were  identified  as  those  intended  for 
general  use  in  developing  computer-mediated  instructional  environments,  those 
specifically  intended  for  developing  instructional  simulations,  and  those  intended  for 
developing  realistic  automated  and  semi-automated  participants  (friendly,  unfriendly,  or 
neutral)  for  simulations. 

As  shown  in  Table  1,  10  SBIR/STTR  topics  were  included  in  the  “general 
authoring  tools”  sub-area.  Eight  of  these  topics  were  presented  at  the  2005  workshop,  and 
two  were  added  later.  Nine  SBIR/STTR  topics  were  included  in  the  “authoring  tools  for 
simulations”  sub-area.  Six  of  these  were  presented  at  the  2005  Workshop,  and  three  were 
added  later.  Two  SBIR/STTR  topics  were  included  in  the  “authoring  tools  for  com¬ 
puter-generated  participants”  sub-area.  Both  of  these  topics  were  presented  at  the  2005 
workshop. 


c.  Simulation-Based  Training 

Since  the  days  of  the  medieval  quintain  (a  device  used  to  train  mounted  knights  in 
proper  techniques  of  the  lance)  and  earlier,  simulators  and  simulation  have  been  used 
widely  to  provide  military  training.  Today’s  concern  in  DoD  with  networked  simulation, 
interoperability,  gaming,  realism,  immersion,  situated  learning,  and  the  like  may  ensure 
the  commercial  value  of  innovative  development  in  this  area. 

Much  DoD  training  and  education  is  conducted  using  simulations  (e.g.,  Andrews 
and  Bell,  2000),  and  there  are  good  reasons  for  this.  Simulations  allow  “situated”  per¬ 
formance-based  training  to  be  conducted  with  reduced  costs  and  increased  safety,  visibil¬ 
ity,  replication,  and  immersion  for  learners.  These  features  permit  practice  and  feedback 
on  instructional  objectives  that  would  otherwise  be  impracticable  to  provide.  SBIR/STTR 
topics  in  training  and  education  are  divided  into  two  groups:  simulation  training  for 
teams  and  simulation  training  for  individuals.  Computer-mediated  learning  environments 
appear  to  be  at  the  core  of  both  groups.  As  noted,  topics  included  for  the  2005 
SBIR/STTR  review  focused  on  supporting  training  and  education  through  the  use  of 
simulation  rather  than  focusing  on  the  computational  techniques  used  to  produce  and 
operate  simulations. 

As  shown  in  Table  1,  nine  SBIR/STTR  topics  were  included  in  the  “simulation 
training  for  teams”  sub-area.  Eight  of  these  topics  were  presented  at  the  2005  workshop, 
and  one  was  added  later.  Seventeen  SBIR/STTR  topics  were  included  in  the  “simulation 


10 


training  for  individuals”  sub-area.  Thirteen  of  these  topics  were  presented  at  the  2005 
workshop,  and  four  were  added  later. 

d.  Training  System  Design  and  Development 

Intelligent  instructional  systems,  authoring  tools,  and  simulation  all  come  together 
under  the  heading  of  training  system  design  and  development.  The  goal  is  to  create  a 
design  capability  that  reliably  ensures  the  accomplishment  of  instructional  objectives  in  a 
cost-effective  manner  by  all  members  of  the  training  audience.  Especially  in  training  sys¬ 
tems  development,  this  goal  is  not  one  of  art  or  even  science  but  is  more  akin  to  engi¬ 
neering  (e.g.,  Woolf  and  Regian,  2000).  Once  the  specifications  for  knowledge,  skills, 
and  abilities  are  articulated  as  instructional  objectives — along  with  their  tasks, 
conditions,  and  standards — the  instructional  engineering  goal  is  to  apply  design 
principles  that  reliably  achieve  these  objectives,  on  time  and  within  budget. 

Many  SBIR/STTR  topics  focused  on  the  development  of  specific  training  sys¬ 
tems.  The  appeal  of  using  SBIR/STTR  resources  to  develop  specific  training  problems  is 
not  difficult  to  understand  and  appreciate.  Among  other  things,  SBIR/STTR  funding  can 
be  used  to  address  and  solve  emerging  or  recalcitrant  training  problems,  thereby  pro¬ 
ducing  products  that  are  likely  to  find  a  waiting  market — at  least  within  DoD.  However, 
topics  included  in  this  review  were  required  to  address  OUSD(S&T)  and  SBIR/STTR 
interests  in  advancing  S&T  training  and  education.  Topics  that  targeted  the  creation  of  a 
specific  training  capability  were  included  in  this  review  and  this  area,  but  they  were 
topics  that  also  seemed  likely  to  advance  training  and  education  S&T. 

As  shown  in  Table  1,  22  SBIR/STTR  topics  were  included  in  the  training  system 
design  and  development  area.  Twenty  of  these  topics  were  presented  at  the  2005  work¬ 
shop,  and  two  were  added  later. 

2,  2005  Workshop  Topic  Presentations 

R&D  managers  who  presented  topics  at  the  June  2005  Workshop  were  asked  to 
follow  a  fairly  tight,  common  format  for  their  presentations.  Appendix  B  shows  this  for¬ 
mat.  It  called  for  brief  statements  of  the  general  problem  or  issue  to  be  addressed  by  the 
topic,  its  objective,  and  the  objectives  and  results  of  Phase  1  projects  completed  under  the 
topic.  Phase  2  results,  final  deliverable(s),  military  payoff,  payoff  to  the  company  per¬ 
forming  the  project,  and  general  lessons  learned  under  the  topic  also  had  to  be  reported, 
along  with  the  topic’s  successes,  payoffs,  and  remaining  R&D  gaps. 
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E.  SBIR/STTR  Investment  in  Training  and  Education 

The  criteria  used  in  this  review  for  the  selection  of  SBIR/STTR  topics  are  fairly 
restrictive.  More  topics  could  reasonably  have  been  included  by  relaxing  these  criteria; 
however,  even  under  these  constraints,  the  SBIR/STTR  program  in  training  and 
education  is  a  sizable  R&D  investment.  It  totals  over  $83  million  across  282  awards 
(184  for  Phase  1  projects  and  98  for  Phase  2  projects). 

Tables  2,  3  and  4  show  Phase  1  awards  (Table  2),  Phase  2  awards  (Table  3),  and 
both  phases  combined  (Table  4).  These  tables  show  the  SBIR/STTR  investment  made  in 
the  80  education  and  training  topics  identified  for  this  review,  by  organization  and  by  FY 
(FY  1999-FY  2004).  FY  2004  SBIR/STTR  topic  awards  are  included  in  these  tables,  but 
funds  awarded  by  these  topics  in  FY  2005  are,  of  course,  not.  A  search  was  made  for 
SBIR/STTR  topics  that  were  initiated  before  FY  1999,  met  all  other  criteria  for  inclusion 
in  this  review,  and  made  awards  in  FY  1999  or  later;  however,  none  were  found. 


Table  2.  Total  Phase  1  Training  and  Education  SBIR/STTR  Projects  (Number)  and 
Award  Amounts  ($000)  by  Service/Organization  and  by  FY 


Fiscal  Year 

Army 

Navy 

Air  Force 

OSD 

DARPA 

Totals 

FY  1 999 

(1) 

99.9 

- 

(6) 

597.6 

(2) 

219.4 

- 

(9) 

916.9 

FY  2000 

(3) 

309.8 

- 

(4) 

396.9 

(6) 

539.6 

- 

(13) 

1,246.3 

FY  2001 

(1) 

120.0 

(7) 

580.0 

(6) 

599.7 

(26) 

2,593.4 

- 

(40) 

3,893.1 

FY  2002 

(7) 

738.3 

(2) 

170.0 

(11) 

1,099.5 

(43) 

4,276.4 

- 

(63) 

6,284.2 

FY  2003 

(8) 

834.2 

(3) 

266.6 

(9) 

899.6 

(13) 

1,292.4 

(7) 

690.2 

(40) 

3,983.0 

FY  2004 

(9) 

829.0 

- 

(4) 

399.6 

(6) 

599.4 

- 

(19) 

1,828.0 

Totals 

(29) 

2,931.2 

(12) 

1,016.5 

(40) 

3,992.8 

(96) 

9,520.6 

(7) 

690.2 

(184) 

18,151.3 

Proportion  of 
Phase  1  Total 

0.16 

0.06 

0.22 

0.52 

0.04 

100.0 

Note  1  for  Table  2:  Sums  (totals)  may  differ  slightly  because  of  rounding. 
Note  2  for  Table  2:  Number  of  projects  are  shown  in  parentheses. 
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Table  3.Total  Phase  2  Training  and  Education  SBIR/STTR  Projects  (Number)  and 
Award  Amounts  ($000)  by  Service/Organization  and  by  FY 


Fiscal  Year 

Army 

Navy 

Air  Force 

OSD 

DARPA 

Totals 

FY  1 999 

- 

- 

- 

- 

- 

- 

FY  2000 

(1) 

500.0 

- 

(3) 

2,794.2 

(4) 

1,730.3 

- 

(8) 

5,024.5 

FY  2001 

- 

- 

(4) 

3,092.2 

(9) 

4,488.7 

- 

(13) 

7,580.9 

FY  2002 

(4) 

3,168.5 

(1) 

721.0 

(3) 

2,249.5 

(12) 

7,993.0 

- 

(20) 

14,132.0 

FY  2003 

(5) 

3,187.8 

(4) 

2,022.9 

(9) 

6,959.3 

(14) 

8,599.2 

- 

(32) 

20,769.2 

FY  2004 

(10) 

7,046.9 

(0) 

(5) 

3,749.8 

(5) 

3,374.9 

(5) 

3,373.7 

(25) 

17,545.3 

Totals 

(20) 

13,903.1 

(5) 

2,743.9 

(24) 

18,845.1 

(44) 

26,186.1 

(5) 

3,373.7 

(98) 

65,052.0 

Proportion  of 
Phase  2  Total 

0.20 

0.05 

0.24 

0.45 

0.05) 

100.0 

Note  1  for  Table  3:  Sums  (totals)  may  differ  slightly  because  of  rounding. 
Note  2  for  Table  3:  Number  of  projects  are  shown  in  parentheses. 


Table  4.  Total  Training  and  Education  SBIR/STTR  Projects  (Number)  and 
Award  Amounts  ($000)  by  Service/Organization  and  by  FY 


Fiscal  Year 

Army 

Navy 

Air  Force 

OSD 

DARPA 

Totals 

FY  1 999 

(1) 

99.9 

- 

(6) 

597.5 

(2) 

219.4 

- 

(9) 

916.8 

FY  2000 

(4) 

809.7 

- 

(7) 

3,191.1 

(10) 

2,269.9 

- 

(21) 

6,270.7 

FY  2001 

(1) 

120.0 

(7) 

579.7 

(10) 

3,692.0 

(35) 

7,082.1 

- 

(53) 

11,473.8 

FY  2002 

(11) 

3,906.8 

(3) 

891.0 

(14) 

3,349.0 

(55) 

12,269.4 

- 

(83) 

20,416.2 

FY  2003 

(13) 

4,022.0 

(7) 

2,289.7 

(18) 

7,858.9 

(27) 

9,891.7 

(7) 

690.2 

(72) 

24,752.5 

FY  2004 

(19) 

7,875.8 

- 

(9) 

4,149.4 

(11) 

3,974.2 

(5) 

3,373.7 

(44) 

19,373.1 

Totals 

(49) 

16,834.2 

(17) 

3,760.4 

(64) 

22,837.9 

(140) 

35,706.8 

(12) 

4,063.9 

(282) 

83,203.2 

Proportion  of 
Grand  Total 

0.17 

0.06 

0.23 

0.50 

0.04 

100.0 

Note  1  for  Table  4:  Sums  (totals)  may  differ  slightly  because  of  rounding. 
Note  2  for  Table  4:  Number  of  projects  are  shown  in  parentheses. 
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Based  on  these  data,  it  appears  that  DoD  SBIR/STTR  investment  in  training  and 
education  has  grown  considerably — from  $0.9  million  in  FY  1999  to  $24.8  million  in 
FY  2003  (Table  4).  The  investment  was  lower  in  FY  2004  ($19.4  million),  but  it  is  still 
greater  by  a  factor  of  about  20  over  FY  1999. 

For  the  FY  1999-FY  2004  time  period,  among  the  Services  (Tables  1  and  4),  the 
Air  Force,  with  23  topics,  64  awards,  and  $23  million  in  awarded  funding,  appears  to 
have  taken  the  most  advantage  of  the  SBIR/STTR  program  to  support  S&T  in  training 
and  education.  The  Army  is  not  far  behind,  with  20  topics,  49  awards,  and  $17  million  in 
awarded  funding.  Navy  participation  produced  5  topics,  17  awards,  and  $4  million  in 
awarded  funding. 

OSD,  specifically  OUSD(S&T)  BioSystems,  is  a  major  player  in  the  SBIR/  STTR 
training  and  education  arena  with  31  topics  (Table  1),  140  projects,  and  $36  million  in 
awarded  funding  (Table  4),  all  defined  and  managed  by  other  DoD  R&D  organizations. 
OSD  training  and  education  SBIR/STTR  activity  accounts  for  about  43  percent  of  all 
funding  in  this  area  during  the  FY  1999-FY  2004  time  period. 

All  three  Services  have  participated  roughly  equally  in  developing  topics  and 
managing  training  and  education  projects  for  the  OSD  SBIR/STTR  program.  Ten  OSD 
topics  were  developed  and  managed  by  Army  R&D  organizations,  10  topics  were  devel¬ 
oped  and  managed  by  Navy  R&D  organizations,  and  8  topics  were  developed  and  man¬ 
aged  by  Air  Force  R&D  organizations.  Three  OSD  topics  were  developed  and  managed 
by  the  Special  Operations  Command  (SOC). 

Table  5  shows  the  SBIR/STTR  investment  in  all  80  education  and  training  topics 
identified  for  this  review  by  organization  and  by  (sub)area.  SBIR/STTR  investments  in 
ITSs  supported  11  topics  (Table  1)  and  awarded  $15  million  to  45  projects  (Table  5). 
Authoring  Tools,  Simulation-Based  Training,  and  Training  System  Design  and  Develop¬ 
ment  were  roughly  equivalent  in  number  of  topics  (21,  26,  and  22,  respectively:  see 
Table  1),  number  of  projects  (81,  76,  and  80,  respectively:  see  Table  5),  and  award 
amount  ($23,  million,  $24  million,  and  $23  million,  respectively:  see  Table  5).  Most 
(about  42  of  80)  SBIR/STTR  topics  focused  on  developing  training  packages  for  specific 
subject  areas. 

Table  6  shows  that  DARPA  and  the  Anny  lead  in  proportion  of  Phase  2  to 
Phase  1  awards  with  7 1  and  69  percent,  respectively,  of  their  Phase  1  awards  progressing 
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Table  5.  Training  and  Education  SBIR/STTR  Projects  (Number)  and 
Award  Amounts  ($000)  by  Organization  and  by  (Sub-)Areas 


R&D  Areas 

Army 

Navy 

Air  Force 

OSD 

DARPA 

Totals 

ITSs 

(8) 

2,738.7 

- 

(4) 

1,910.0 

(21) 

5,912.6 

(12) 

4,063.9 

(45) 

14,625.2 

Authoring  Tools 

General  authoring  tools 

(2) 

849.0 

(5) 

1,030.8 

(2) 

849.9 

(37) 

8,888.6 

- 

(46) 

11,618.3 

Authoring  tools  for 
simulations 

(4) 

986.6 

(4) 

641.8 

(8) 

3,399.0 

(12) 

3,135.1 

- 

(28) 

8,162.5 

Authoring  tools  for 

computer-generated 

participants 

(2) 

849.9 

(5) 

1,896.3 

(7) 

2,746.2 

Simulation-Based  Training 

Simulation  training  for 
teams 

(10) 

3,870.9 

- 

(13) 

5,042.4 

(6) 

599.4 

- 

(29) 

9,512.7 

Simulation  training  for 
individuals 

(15) 

4,473.7 

(1) 

100.0 

(24) 

6,993.6 

(7) 

1,953.0 

- 

(47) 

13,520.3 

Training  Systems  Design 
and  Development 

(8) 

3,065.3 

(7) 

1,987.8 

(8) 

2,746.8 

(57) 

15,218.1 

- 

(80) 

23,018.0 

Totals 

(49) 

16,834.4 

(17) 

3,760.4 

(64) 

22,837.9 

(140) 

35,706.7 

(12) 

4,063.9 

(282) 

83,203.6 

Note  1  for  Table  5:  Sums  (totals)  may  differ  slightly  because  of  rounding. 
Note  2  for  Table  5:  Number  of  projects  are  shown  in  parentheses. 


Table  6.  Proportion  of  Phase  2  to  Phase  1  Project  Awards  by  Service/Organization 


Army 

Navy 

Air  Force 

OSD 

DARPA 

Totals 

Proportions 

(20/29) 

0.69 

(5/12) 

0.42 

(24/40) 

0.60 

(44/96) 

0.46 

(5/7) 

0.71 

^mT!|cISK|n 

to  Phase  2.  The  Air  Force  is  fairly  close  with  60  percent.  Both  OSD  and  Navy  Phase  1 
projects  are  less  likely  to  progress  to  Phase  2,  with  46  and  42  percent,  respectively.  Set¬ 
ting  aside  this  variance,  it  appears  overall  that  about  half  (53  percent)  of  Phase  1  projects 
in  SBIR/STTR  training  and  education  progress  to  Phase  2. 

Using  SBIR/STTR  funds,  then,  to  develop  specific  training  packages  (both 
training  systems  and  simulations)  appears  to  be  a  major  factor  in  SBIR/STTR  investment. 
Many  candidate  topics  were  eliminated  because  they  appeared  to  be  more  focused  on 
producing  a  simulator,  simulation,  or  training  system  to  solve  a  specific  training  problem 
than  on  advancing  S&T  in  training  and  education.  Those  topics  that  remain  advance,  at 
least  to  some  extent,  the  state  of  the  art  and  practice  in  training  and  education.  Continued 
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effort  appears  to  be  needed  to  ensure  that  SBIR/STTR  topics  and  projects  in  training  and 
education  stimulate  technological  innovation,  in  accord  with  the  objectives  of  SBIR/ 
STTR  legislation. 

F.  SBIR/STTR  Training  and  Education  S&T  Products 

The  products  from  DoD  SBIR/STTR  investments  in  training  and  education  dis¬ 
cussed  in  this  section  are  matched  with  the  topic  area  into  which  they  seem  to  fit  best. 
This  categorization  was  based  on  a  reading  of  product  reports  and  topic  descriptions. 
That  said,  however,  some  products  can  easily  be  seen  to  be  relevant  to  more  than  one 
topic.  The  purpose  of  this  section  is  to  suggest  how  all  these  products  might  be 
aggregated  into  general  areas  of  S&T  activity  and  to  highlight  the  quality,  productivity, 
and  scope  of  effort  evident  in  the  FY  1999-FY  2004  DoD  SBIR/STTR  investments  in 
training  and  education. 

1.  ITSs 

A  review  of  ITS  activities  in  DoD  SBIR/STTR  training  and  education  topics  and 
projects  suggests  four  general  S&T  directions  and  concerns: 

1 .  Natural  language  tutorial  dialogue 

2.  Automated,  continuous,  and  unobtrusive  assessment  of  learner  state  and 
progress 

3.  Representation  of  knowledge,  skill,  and  ability 

4.  Dynamic  assembly  of  training  from  instructional  objects  to  help  learners  pro¬ 
gress  from  a  current  state  of  knowledge,  skill,  and  ability  to  the  objective 
state. 

Among  the  capabilities  DoD  SBIR/STTR  training  and  education  topics  and  pro¬ 
jects  produced  in  the  area  of  ITSs  in  the  FY  1999-FY  2004  time  period  are  the  following: 
•  Natural  Language  Tutorial  Dialogue 

-  Latent  Semantic  Analysis  (LSA)  to  match  computer-generated  interac¬ 
tions  with  the  semantic  structure,  content,  and  level  of  discourse  of 
learners’  notes  and  textual  contributions  in  training  for  operations  plan¬ 
ning  and  leadership 

-  Agent  system  capable  of  engaging  in  natural  language  tutorial  dialogue 
for  coaching  and  training  for  operations  execution 

-  Dialogue -based,  goal-directed  practice  to  improve  command  reasoning. 
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•  Automated,  Continuous,  and  Unobtrusive  Assessment  of  Learner  State  and 

Progress 

-  Overlay  techniques  for  matching  learner  knowledge  state  with  automati¬ 
cally  generated  “expert”  solutions  for  training  for  operations  execution 

-  Automated  observer/controller  (O/C)  to  provide  individualized  guidance 
and  coaching  during  computer-based  exercise/simulations  and  diagnostic 
After-Action  Review  (AAR) 

-  Lightweight,  low-cost  brainwave  detection  of  attention  and  focus  during 
training 

-  Automated  analysis  of  facial  expressions  to  assess  learner  levels  of  atten¬ 
tion,  mental  effort,  confusion,  and  fatigue 

-  Low-cost,  high-performance,  minimally  invasive  eye  tracking  to  assess 
learner  perfonnance  and  prescribe  optimal  training  experience 

-  Automated  recognition  and  reporting  of  key  situations  and  actions  in 
simulation-based  teamwork  training  using  massively  multiplayer  (MMP) 
gaming  environments  to  train  small  dismounted  teams  for  urban  envi¬ 
ronment  operations. 

•  Representation  of  Knowledge,  Skill,  and  Ability 

-  Intelligent  tutoring  linked  with  simulation-based  training  for  training 
military  medical  teams  to  make  correct  critical  decisions  under  time- 
limited  conditions 

-  Recommendations  for  granularity  in  expert  and  student  models  to  sup¬ 
port  training  for  anti-submarine  warfare  tactical  air  controllers 

-  Support  for  information  operations  and  information  warfare  training 
using  cognitive  modeling,  knowledge  engineering,  and  formal  represen¬ 
tations  of  knowledge. 

•  Dynamic  Assembly  of  Training  From  Instructional  Objects 

-  LSA  techniques  for  locating  in  large  databases  information  relevant  to 
learners’  needs  in  training  for  operations  execution  and  leadership 

-  Selection  of  assets  and  construction  of  Web  pages  based  on  learner 
model  generated  to  support  training  and  performance  aiding  at  a  distance 

-  Novel  ways  to  create  face-to-face  interaction,  coaching,  and  feedback 
and  provide  rapid  access  to  expertise. 
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2.  Authoring  Tools 

A  review  of  authoring  tools  produced  by  the  DoD  SBIR/STTR  training  and  edu¬ 
cation  topics  and  projects  suggested  five  general  S&T  directions  and  concerns: 

1 .  Linking  design  and  development  to  instructional  objectives 

2.  Authoring  by  non-IT  specialists 

3.  Simulation  and  scenario  generation  and  preparation 

4.  Rapid  (agile)  simulation  and  scenario  authoring 

5.  Creation  of  computer-generated  (automated  and  semi-automated) 

participants  (allies,  enemies,  and  neutrals). 

Among  the  authoring  tools  DoD  SBIR/STTR  training  and  education  topics  and 
projects  produced  in  the  FY  1999-FY  2004  time  period  are  the  following: 

•  Linking  Design  and  Development  to  Instructional  Objectives 

-  Authoring  guidelines  for  incorporating  multisensory  (visual,  auditory, 
haptic,  tactile,  olfactory,  and  so  forth)  components  in  training  for  infor¬ 
mation  processing  and  fusion  in  command  and  control  (C2) 
environments 

-  Authoring  tools  for  combining  low-fidelity  production  with  high  cogni¬ 
tive  fidelity  to  produce  training  vignettes  that  focus  attention  on  specific 
training  objectives  and  sharing  of  expertise 

-  Authoring  tools  to  embed  problem-based  training  capabilities  in 
emerging  Objective  Force  equipment 

-  Authoring  tools  to  ensure  motivation  and  interest  in  (distributed)  Web- 
based  training  material  through  the  inclusion  of  social  interaction, 
dynamic  graphics,  sound  effects,  and  gaming/simulation. 

•  Authoring  by  Non-IT  Specialists 

-  Fully  configurable  user  interface  for  instructor-designed  simulations  and 
scenarios  involving  three-dimensional  (3-D)  displays  for  satellite  opera¬ 
tions  training 

-  Cognitive  analysis  and  framework  leading  to  authoring  tools  that  allow 
nondesigner  SMEs  to  create  Distance  Learning  materials 

-  Diagnostic  assessment  of  Navy  watch  team  readiness  linked  to  recom¬ 
mended  training  interventions  that  address  performance  shortfalls 

-  Authoring  tools  for  SMEs  to  provide  and  support  a  continuing  cycle  of 
analysis  and  development 
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-  Authoring  tools  to  allow  SMEs  to  design  and  create  case-based  instruc¬ 
tion  that  help  students  abstract  and  apply  principles  from  multiple  cases. 

Simulation  and  Scenario  Generation  and  Preparation 

-  Development  and  validation  of  a  mission-oriented  process  to  determine 
the  training  impacts  of  simulation  federation  modifications  on  proposed 
system  enhancements 

-  Tools  for  diagnostic  assessment  of  training  scenarios  in  achieving 
training  objectives 

-  Tools  to  support  Distributed  Mission  Operations  (DMO)  training  by  cus¬ 
tomizing  fidelity  requirements  to  mission  requirements  and  assessing  the 
value  of  these  customized  levels  of  fidelity  in  enabling  learners  to  meet 
training  objectives 

-  Tools  to  link  mission-essential  competencies  to  knowledge  and  skills 
that  inform  the  automated  and  semi-automated  selection  of  training 
objectives  and  the  design  of  scenarios 

-  Architecture  and  authoring  tools  for  integrating  3-D  components  with 
Sharable  Content  Objects  (SCOs)  to  create  Advanced  Distributed 
Learning  (ADL)-conformant  training  scenarios. 

Rapid  (Agile)  Simulation  and  Scenario  Authoring 

-  Authoring  tools  for  materials  that  allow  learners  to  collect  and  integrate 
infonnation  from  a  variety  of  sources  in  making  tactical  decisions 
rapidly 

-  Authoring  tools  for  creating  and  customizing  scenarios  that  are  directly 
linked  to  immediate  training  objectives  and  can  be  modified  in  near-real 
time 

-  Authoring  tools  to  enhance  the  agility  of  scenario  creation  and  reduce 
development  time  to  a  matter  of  days  or  less. 

Creation  of  Computer-Generated  Participants 

-  Authoring  tools  for  representing  culture -based  behavior — including 
speech-  and  gesture -based  communications — in  joint,  multinational 
operations  in  asymmetric  environments 

-  Authoring  tools  for  incorporating  behavior  moderators  into  materials  for 
crowd-control  training. 


3.  Simulation-Based  Training 

A  review  of  simulation  products  produced  by  the  DoD  SBIR/STTR  training  and 
education  topics  and  projects  suggested  five  general  S&T  directions  and  concerns: 

1.  Simulation  training  linked  to  instructional  objectives  (for  individuals) 

2.  Simulation  training  linked  to  instructional  objectives  (for  teams) 

3.  Post-simulation  performance  feedback 

4.  Synthetic  agents  and  avatars  in  simulation 

5.  Haptics  in  medical  training. 

Among  the  capabilities  DoD  SBIR/STTR  training  and  education  topics  and  pro¬ 
jects  produced  in  the  area  of  Simulation-Based  Training  in  the  FY  1999-FY  2004  time 
period  are  the  following: 

•  Simulation  Training  Linked  to  Instructional  Objectives  (for  Individuals) 

-  Web-based  training  to  enhance  cultural  adaptability  in  leaders  and  sol¬ 
diers,  with  an  administration  system  for  continued  maintenance,  modifi¬ 
cation,  and  development 

-  Visual,  agent-based  representation  for  training  and  perfonnance  aiding 
in  employing  parameterized,  hierarchical,  and  conditional  procedures 
used  to  communicate  with  satellites  in  low  earth  orbit  (LEO) 

-  System  for  training  in  the  use  of  handheld  landmine  detection  systems  to 
recognize  signatures  of  landmine  and  clutter  types  under  a  variety  of  soil 
and  environmental  conditions 

-  Computer-based,  Sharable  Content  Object  Reference  Model  (SCORM)- 
compliant  systems  to  provide  initial,  advanced,  and  refresher  training  in 
medical  emergency  technical  skills  that  integrate  first  responder  higher 
level  cognitive  skills  with  task  work  skills 

-  Personal  computer  (PC)-based  simulation  of  air  campaigns  for  training 
Joint  Force  Air  Component  Commanders  (JFACCs)  in  mission-essential 
and  supporting  competencies. 

•  Simulation  Training  Linked  to  Instructional  Objectives  (for  Teams) 

-  Team  training  for  space  operations  using  (1)  distributed  simulation, 
(2)  collaborative  tools  to  provide  workflow  modeling,  process  control, 
data  mediation,  and  communications  infrastructure  combined  with  intel¬ 
ligent  agents,  and  (3)  advanced  human-machine  technologies  with  auto¬ 
mated  script  creation  and  approval 
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-  Deployable  trainer  using  simulated  team  members,  team  dialogue,  and 
support  tools  to  coach  medical  professionals  in  the  decision-making  and 
team-coordination  capabilities  needed  to  meet  major  chemical,  biologi¬ 
cal,  and  radiological  (CBR)  emergencies. 

Post-Simulation  Perfonnance  Feedback 

-  Automated  tools  for  skills  assessment  and  AAR  for  detecting  significant 
events  and  decisions  in  virtual  Distributed  Interactive  Simulation/High 
Level  Architecture  (DIS/HLA)  environments  that  provide  small  dis¬ 
mounted  unit  training  for  military  operations  in  urban  terrain 

-  Methods  for  decomposing  scenarios  into  meaningful  “chunks”  that  pro¬ 
vide  scenario  trigger  identification  and  measurement-based  evaluation  of 
air  combat  perfonnance  for  air  combat  scenarios 

-  Predictive  competency-based  measurement  and  performance-aiding  sys¬ 
tem  that  integrates  observer-based  and  simulator-based  measures  of 
complex  perfonnance  in  environments  such  as  DMO  training  to  pennit 
diagnosis  and  prescription  in  distributed  learning 

-  Cognitive  models  and  workload  assessment  for  specifying  and  assessing 
perfonnance  in  a  satellite  command  generation  capability 

-  Perfonnance  support  toolkit  to  assess  satellite  operation  work  domains 
and  identify  satellite  operator  work,  performance,  and  context-appropri¬ 
ate  feedback  and  support 

-  Tools  and  techniques  for  automating  the  unobtrusive  and  automated 
assessment  of  soldier  situational  awareness  based  on  movement  and 
behavior  in  virtual  simulations. 

Synthetic  Agents  and  Avatars  in  Simulation 

-  Development  of  synthetic  agents/team  members  playing  different  roles 
to  provide  distributed,  anywhere,  anytime,  interactive,  time-stressed 
decision-making  training  and  personnel  tracking  for  Air  Operations 
Center  (AOC)  personnel 

-  Development  of  an  ontology  for  synthetic  agents  who  collaborate  with 
space  crew  members  using  speech  recognition  and  simulated  equipment 
to  provide  distributed  training  in  the  use  of  subscriber  tenninals 

-  Low-cost,  commercial  off-the-shelf  (COTS)-based,  PC  undergraduate 
pilot  training  system  with  agent-enabled  synthetic  tower  controller  and 
tutor  for  ground-based  training  in  landing  patterns  and  radio  procedures 
during  traffic  pattern  operations 


-  Tutoring  system  with  virtual  characters  to  support  the  acquisition  of  the 
social  engineering  skills  needed  to  obtain  intelligence  by  infiltrating  and 
gaining  access  to  an  objective  organization. 

•  Haptics  in  Medical  Training 

-  Application  of  a  haptic  mouse  for  training  in  patient  palpitation  and  rec¬ 
ognition  of  arterial  pulses 

-  Virtual  free-floating  needle  with  multitactile  force  feedback  for  needle 
insertion  training  in  thoracentesis  (a  procedure  to  remove  fluid  from  the 
space  between  the  lining  of  the  outside  of  the  lungs  (pleura)  and  the  wall 
of  the  chest) 

-  Open  surgical  skills  simulator  that  employs  haptic  feedback  and  a  cur¬ 
riculum  based  on  task  analyses  of  surgical  procedures  and  perfonnance 
metrics 

-  Hybrid  manikin  simulator  combined  with  a  virtual  reality  (VR)  curricu¬ 
lum  adaptable  to  individual  learner  needs  and  performance  metrics  for 
training  in  procedures  used  to  control  limb  hemorrhage 

-  Mock  endotracheal  tube  with  haptic  feedback  coupled  to  a  computer- 
controlled  mannequin  head  with  mechanically  controlled  airway 
geometry 

-  Optimal  approach  for  assessing  simulators  for  training  in  combat  casu¬ 
alty  care  procedures  that  would  lead  to  a  standardized  assessment 
methodology 

-  Metrics-based  training  system  that  combines  tissue  and  blood  ves¬ 
sel/flow  models,  visual  and  haptics  modeling  of  six  primary  surgical 
tools  and  a  graphics  system  in  a  simulator  incorporating  proper  physics 
and  stereo  visualization 

-  Open  surgery  trainer  that  allows  visual  tracking,  free-floating  Magnetic 
Levitation  (Maglev)  haptic  feedback  with  real  surgical  tools  and  sce¬ 
nario-based  training  that  can  be  configured  using  the  extensible  Markup 
Language  (XML) 

-  PC-based  endoscopic  surgical  simulation  system  that  integrates  physics 
processing,  device  tracking,  and  graphic  rendering  with  electronic  cir¬ 
cuitry  that  identifies  and  responds  to  the  insertion  of  tools  of  different 
diameters  and  a  gas-pressure  approach  to  simulate  syringe  haptics. 
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4.  Training  System  Design  and  Development 

A  review  of  training  system  design  and  development  products  produced  by  the 
DoD  SBIR/STTR  training  and  education  topics  and  projects  suggested  three  general  S&T 
directions  and  concerns: 

1.  Team  training 

2.  Cognitive  readiness  training  for  leaders 

3.  Training  in  areas  of  special  concern. 

Among  the  capabilities  DoD  SBIR/STTR  training  and  education  topics  and  pro¬ 
jects  produced  in  the  area  of  Training  System  Design  and  Development  in  the  FY  1 999— 
FY  2004  time  period  are  the  following: 

•  Team  Training 

-  Scenario-  and  case-based  training  in  an  online  environment  that  allows 
connectivity  among  trainees,  instructor  monitoring,  practice,  and  feed¬ 
back  to  promote  shared  goals,  beliefs,  expectations,  and  intentions 
within  command  group  planning  teams 

-  Web-based,  scenario-based  distributed  training  using  video  clips  and 
automated  scoring  logic  to  leverage  DMO  technology  and  enhance  deci¬ 
sion-making  by  quickly  configured,  rapidly  deployed  teams  such  as  the 
United  States  Marine  Corps  (USMC)  Combined  Arms  teams  and  United 
States  Air  Force  (USAF)  Expeditionary  Medical  Support  (EMEDS) 
teams. 

•  Cognitive  Readiness  Training  for  Leaders 

-  Comprehensive  frameworks,  including  specific  critical  skills  for 
successful  critical  thinking  and  leader  dialogue  skills  needed  by  battle 
command  (BC)  teams  (to  improve  memory  for  messages  and  elicit  evi¬ 
dence-based  inferencing  and  questioning  of  information) 

-  Web-based  assessment  and  training  with  automated  scenario  structuring 
and  selection  for  enhancing  the  ability  of  team  leaders  to  recognize  and 
act  on  critical  points  likely  to  affect  the  performance  of  multicultural, 
multinational  teams 

-  Customizable  training  and  self-assessment  program  for  leader  self¬ 
development  using  self-appraisal,  self-regulation,  and  self-development 
learning  contracts 

-  Simulation  environment  for  training  JFACCs  based  on  an  analyses  of 
the  knowledge  resources  available  to  support  cognitive  skills  identified 
in  cognitive  task  analysis  of  senior  leader  decision  and  assessments 
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-  Application  and  extension  of  DARPA’s  Decision  Net  technology  to 
enhance  the  situation  awareness  (SA)  (perceive  critical  elements,  assess 
their  relative  importance,  and  project  future  events)  of  small-unit  leaders 
in  Web-based  distributed  learning 

-  Web-based  training  to  enhance  junior  officer  leadership  skills  in 
communication,  team-building,  critical  thinking,  social  appraisal,  and 
emotional  management 

-  Scenario-based  practice  for  intelligence  analysts  focused  on  context-spe¬ 
cific  analytical  skills,  meta-cognitive  skills  to  probe  critical  thinking, 
intelligence  research,  and  automated  assessment  to  improve  report  gen¬ 
eration  skills. 

•  Training  in  Areas  of  Special  Concern 

-  Training  system  employing  closed-loop,  adaptive  technology  as  the  ena¬ 
bling  technology  for  supporting  training  in  a  reduced  manning 
environment 

-  Validated  naturalistic  decision-making  scenarios  based  on  cognitive  task 
analyses  for  training  multiple  personnel  in  multiple  locations  to  maintain 
SA  and  mission  focus  and  envision  exit  strategies  in  Operations  Other 
Than  War  (OOTW)  (e.g.,  humanitarian  relief,  noncombat  evacuations, 
peacekeeping) 

-  SCORM-confonnant,  PC-based  integrated  operator  training  and 
rehearsal  environment  used  to  provide  emergency  procedures  closed- 
loop  adaptive  training  for  ground  satellite  operators  controlling  multiple 
satellites 

-  SCORM-compliant,  open  system  with  an  underlying  process  ontology 
for  conducting  Web-based  training  in  deficiency  reporting  and 
resolution 

-  Modular,  computer-supported  training  system  based  on  fonnal  analysis 
and  definition  of  expert  sea-keeping  skills  to  enhance  boat  handling 
safety,  mitigate  shock,  and  improve  high-speed  maneuvering  in  heavy 
wave  conditions 

-  Standardized  emergency  medical  protocols  for  emergency  responders 
and  ADL/SCORM-compliant  Web-based  sustainment  training  covering 
knowledge,  application,  and  practice  elements  and  including  perform¬ 
ance  aids  that  can  be  downloaded  to  Palm  devices 

-  Rapid  retraining  or  refresher  training  for  combat  casualty  care  that  can 
be  provided  in  a  variety  of  computer-based  environments:  on  the  Web, 
in  transit,  in  the  field. 
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5.  Across-Topic  Areas 

The  review  identified  goals  that  were  not  limited  to  a  single  area  and  were,  in  dif¬ 
ferent  ways  but  to  an  appreciable  extent,  addressed  across  the  four  topic  areas.  These 
goals  are  as  follows: 

•  Maintain  focus  on  military  relevance 

•  Increase  accessibility  of  training,  education,  and  performance  aiding 

•  Incorporate  “intelligent”  computational  capabilities 

•  Increase  agility,  rapidity,  and  ease  of  use  in  development 

•  Provide  for  reuse  and  interoperability 

•  Develop  instructional  interventions  related  to  training  and  education 
objectives 

•  Develop  simulation  and  games  for  training  and  education 

•  Tailor  training  and  education  to  learner  capabilities  and  needs. 

G.  S&T  Progress  Since  FY  1999 

FY  1999  was  selected  as  the  beginning  year  for  topics  to  include  in  this  effort 
because  of  the  review  (see  Section  D)  of  training  and  education  R&D  performed  in  that 
year.  The  FY  1999-FY  2004  SBIR/STTR  topics  have  been  categorized  in  much  the  same 
way  as  the  efforts  were  reviewed  in  1999.  However,  the  SBIR/STTR  topics  included  in 
this  review  generally  reflect  the  applied  and  entrepreneurial  nature  of  the  SBIR/STTR 
program  rather  than  a  concentration  on  S&T  progress.  They  also  reflect  the  specific 
exclusions  used  to  select  topics  for  this  SBIR/STTR  review. 

Tables  7-13  list  the  1999  issues  and  then  assesses  the  attention  paid  to  them  by 
the  80  SBIR/STTR  training  and  education  topics  included  in  this  review.  Issues  raised  in 
the  1999  review  were  rated  as  Green,  Yellow,  or  Red — roughly  signifying  the  following: 

•  Green  (G):  Satisfactory  current  and/or  anticipated  research  progress 

•  Yellow  (Y):  Borderline  current  and/or  anticipated  research  progress 

•  Red  (R):  Insufficient  current  and/or  anticipated  research  progress. 

The  headings,  R&D  objectives,  and  ratings  in  Tables  7-13  are  taken  directly  from 
the  1999  summary  of  research  objectives.  Products  from  the  FY  1999-FY  2004  SBIR/ 
STTR  training  and  education  topics  are  reflected  onto  the  1999  objectives.  Ratings  of 
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Table  7.  ITSs:  Cognitive  Theory 


1999  R&D  Objective 

1999  Rating 

SBIR/STTR  Rating 

Understand  the  evolution  of  expertise  in  complex,  ill-structured 
environments 

R 

M 

Investigate  domain-specific  problem-solving  skills  and  generaliz- 
able  strategies 

Y 

M 

Determine  the  role  and  significance  of  flexible  and  adaptive 
learning  in  promoting  better  problem  solving  and  critical  thinking 

R 

M 

Capture  effective  behaviors  of  outstanding  human  instructors 

G 

L 

Develop  theoretical  basis  for  employing  interactive  simulations 
and  associated  training  methodologies  to  guide  use  of  simulations 

Y 

L 

Develop  an  understanding  of  the  role  of  interaction  and  collabora¬ 
tion  in  learning 

Y 

M 

Develop  techniques  for  assessing  cognitive  workload  and  strate¬ 
gies  for  mitigating  its  adverse  effects 

Y 

L 

Table  8.  ITSs:  Assessment 


1999  Research  Objective 

1999  Rating 

SBIR/STTR  Rating 

Develop  technologies  allowing  free-form  inputs  for  assessment 

Y 

M 

Generate  valid,  unobtrusive,  near-real-time  assessments  from 
learner  interactions  with  the  learning  environment 

R 

H 

Develop  comprehensive  models  and  measures  of  individual  and 
team  capabilities  and  performance 

R 

M 

Integrate  existing  mission  and  occupational  performance  require¬ 
ments 

Y 

H 

Model  the  relationship  between  individual  training/experience  and 
ease  of  learning  and  retention  of  needed  task-specific  knowledge 
and  skills 

R 

L 

Table  9.  ITSs:  Collaborative  Group  and  Team  Learning 


1999  Research  Objective 

1999  Rating 

SBIR/STTR  Rating 

Define  models  for  collaboration  and  interaction  considering  dis¬ 
tance,  content,  roles,  capabilities,  and  task  requirements 

Y 

H 

Create  and  evaluate  collaboration  arrangements  and  interaction 
strategies 

G 

M 

Develop  team-level  tutoring  concepts 

Y 

H 

Table  10.  ITSs:  Intelligent  Tutoring 


1999  Research  Objective 

1999  Rating 

SBIR/STTR  Rating 

Determine  relevant  task  characteristics  for  instructional  design 
and  selection  of  instructional  strategies 

Y 

H 

Ascertain  relevant  individual  characteristics  for  instructional  design 
and  selection  of  instructional  strategies 

G 

M 

Examine  interactions  between  task  and  individual  characteristics 

Y 

M 

Develop  a  tutoring  capability  sensitive  to  subject  matter,  level  of 
expected  expertise,  and  learner  motivation,  ability,  and 
preparation 

Y 

M 

Model  effective  human  tutorial  dialogue 

G 

M 

Identify  optimal  instructional  strategies  and  technical  approaches 
to  natural  language  understanding,  generation,  and  dialogue  man¬ 
agement 

Y 

M 

Develop  tools  for  authoring  and  modifying  natural  language  dia¬ 
logue 

Y 

L 

Develop  hardware  and  associated  displays  for  augmented  reality 
systems 

G 

L 

Table  11.  Authoring  Tools 


1999  Research  Objective 

1999  Rating 

SBIR/STTR  Rating 

Create  authoring  tools  for  curriculum,  simulations,  assessment, 
system  management,  and  intelligent  tutors 

Y 

H 

Provide  automated  feedback  of  individual  and  system 
performance  data  to  centralized  facilities 

G 

L 

Develop  reusable  components  of  intelligent  computer-assisted 
instruction  and  performance  coaches 

Y 

M 

Develop  intelligent  search  engines  for  quickly  selecting,  parsing, 
and  reusing  archival  knowledge 

Y 

M 
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Table  12.  Distributed  Simulation  Environments  for  Instruction 


1999  Research  Objective 

1999  Rating 

SBIR/STTR  Rating 

Develop  rapid,  efficient  processes  and  procedures  for  verification 
and  validation  of  ADL  environments 

Y 

L 

Enable  synthetic  characters  to  respond  to  verbal  and  nonverbal 
commands  and  actions 

Y 

H 

Enable  the  dynamic  control  of  synthetic  characters  by  instructors 
or  intelligent  tutors 

G 

M 

Enable  the  interchange  of  real  and  virtual  team  members  to  sup¬ 
port  anywhere/anytime  training  delivery 

G 

H 

Develop  models  for  immersive  training  and  education 

Y 

M 

Table  13.  Dynamic  Learning  Management 


1999  Research  Objective 

1999  Rating 

SBIR/STTR  Rating 

Develop  reliable  learner/user  identification,  authentication,  and 
authorization 

G 

L 

Manage  restrictions  of  access  to  the  network  and  maintain 
security  between  source  and  destination 

G 

L 

Provide  data  security  protection  transparently  across  multiple  net¬ 
works  and  organizations 

G 

L 

Automated  indexing  and  searching,  extraction  and  recoding  of 
existing,  knowledge-bearing  digital  data 

Y 

L 

Efficient  methods  for  extracting  and  coding  expert  human  knowl¬ 
edge 

Y 

M 

Learning  content  aggregation  and  disaggregation 

G 

M 

Address  graceful  degradation  of  learning  content 

Y 

L 

High-bandwidth  ubiquitous  network 

G 

L 

contributions  made  by  the  FY  1999-FY  2004  products  to  these  objectives  are  not 
intended  to  be  ratings  of  progress  like  the  Green,  Yellow,  Red  ratings  assigned  to  the 
1999  objectives.  Rather,  they  are  informal  judgments  of  the  concentration  and  attention 
that  the  SBIR/STTR  topics  and  products  reviewed  in  this  report  paid  to  the  1999  R&D 
objectives.  These  ratings  are  as  follows: 

•  High  (H):  significant  attention 

•  Medium  (M):  occasional  attention 

•  Low  (L):  little  or  no  attention. 
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If  we  equate  G  (Green  -  satisfactory  progress)  with  H  (High  -  significant  atten¬ 
tion),  Y  (Yellow  -  borderline  progress)  with  M  (Medium  -  occasional  attention),  and 
R  (Red  -  insufficient  progress)  with  L  (Low  -  little  or  no  attention),  we  find  increased 
attention  from  the  SBIR/STTR  topics  for  about  10  of  the  1999  research  objectives  and 
decreased  attention  for  about  19  of  the  1999  research  objectives. 

At  least  two  comments  apply  to  these  findings.  First,  these  findings  are  based  on 
informed  judgments,  but  they  are  still  judgments.  Second,  while  there  are  similarities  in 
the  incentives  and  purposes  of  the  SBIR/STTR  programs  with  those  of  R&D  endeavors 
in  general,  there  are  significant  differences,  not  the  least  of  which  stem  from  the  applied, 
short-tenn,  and  commercialization  goals  of  the  SBIR/STTR  programs.  However,  even 
under  these  constraints,  areas  can  be  identified  in  which  more  work  is  needed  to  carry  out 
goals  implicit  in  the  overall  SBIR/STTR  enterprise  in  education  and  training. 

H.  S&T  Gaps 

Government  R&D  managers  at  the  June  2005  Workshop  were  asked  to  identify 
gaps — the  remaining  capabilities  needed  to  achieve  their  topic  goals.  Their  views,  along 
with  others  identified  in  personal  discussions  and  still  others  extracted  from  direct  review 
of  SBIR/STTR  technical  documents,  are  summarized  in  the  following  discussion. 

The  gaps  discussed  here  are  genuine  and  credible,  but  the  list  is  undoubtedly 
incomplete.  Moreover,  these  gaps  represent  real  R&D  challenges.  Resolving  them  may 
require  innovation  and/or  technical  breakthroughs,  the  availability  of  which  may  be  as 
much  a  matter  of  serendipity  as  of  deliberate  design.  The  highest  priority  gap  to  fill,  then, 
may  not  be  determined  on  an  a  priori  basis.  It  may  simply  be  the  one  matched  with  the 
most  promising  technical  idea.  The  S&T  gaps  summarized  here  are  organized  around  the 
four  major  topic  areas  used  elsewhere  in  this  report. 

Some  of  these  gaps  may — not  surprisingly — apply  to  more  than  one  of  the  four 
R&D  topic  areas.  Work  to  fill  some  of  the  gaps  has  begun.  Also,  non-SBIR/STTR  work 
to  fill  them  may  be  ongoing.  Nevertheless,  some  next  steps  and  continued  effort  appropri¬ 
ate  for  SBIR/STTR  support  may  be  in  order.  Additional  unresolved  but  significant  S&T 
issues  that  deserve  mention,  may  well  occur  to  readers.  Other  issues  will  arise  in  the 
course  of  S&T  progress. 
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1.  Discussion 


a.  ITSs 

R&D  needs  in  the  ITS  area  fall  under  three  general  topics: 

1 .  Natural  language  dialogue 

2.  Learner  and  expertise  modeling 

3.  Generating  tailored  instructional  interactions. 

More  specific  R&D  efforts  to  advance  S&T  in  each  of  these  three  general  topics 
include  the  following  suggestions: 

•  Natural  Language  Dialogue 

-  Extend  LSA  capabilities  beyond  single  words  to  phrases 

-  Develop  a  grammar  for  tutorial  discourse  to  extend  language  dialogue 
understanding  beyond  single  sentences  or  utterances  to  full  discourse, 
including  anaphoric  references 

-  Disambiguate  semantic  classes  by  creating  topic  maps  and  hierarchies 

-  Include  gestures  as  well  as  speech  and  text  in  mixed-initiative  dialogue. 

•  Learner  and  Expertise  Modeling 

-  Develop  nonintrusive,  continual  assessments  to  construct  robust  learner 
models  from  tutorial  interactions 

-  Create  robust  models  of  the  knowledge,  skills,  and  abilities  shared  by 
members  of  crews,  teams,  and  units 

-  Develop  techniques  for  the  optimal  assembly  of  available  personnel  into 
teams  for  accomplishing  specific  missions  and  tasks 

-  Identify  and  specify  the  granularity  needed  in  student  models  to  achieve 
specific  training,  education,  and  perfonnance-aiding  objectives 

-  Enhance  modeling  capabilities  for  “higher  order”  cognitive  capabilities 
such  as  analysis,  critical  thinking,  adaptability,  creativity,  and  flexibility 

-  Develop  models  of  enabling,  or  “mediating,”  knowledge,  skills,  and 
abilities  as  well  as  those  for  experts 

-  Model  O/C  knowledge,  skill,  and  ability  requirements,  especially  the 
cognitive  processes  and  critical  thinking  skills  (CTSs)  needed  for  AARs 

-  Exploit  the  display,  timing,  and  data-capture  techniques  of  computer 
technology  to  devise  more  comprehensive  models  of  cognition 
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-  Enhance  automated  processes  for  capturing  expert  knowledge,  skills, 
and  abilities. 

•  Generating  Tailored  Instructional  Interactions 

-  Develop  capabilities  to  identify,  collect,  and  assemble  sharable  objects 
in  real  time,  on-demand  for  training  interventions  tailored  to  learner 
needs  and  objectives 

-  Increase  the  precision  with  which  models  of  learner  and  expert  knowl¬ 
edge,  skills,  and  abilities  are  applied  to  tailor  instructional  interventions 

-  Increase  the  efficiency  and  reliability  with  which  instructional 
techniques  lead  to  the  attainment  of  instructional  objectives. 

b.  Authoring  Tools 

R&D  needs  in  the  authoring  tools  area  fall  under  four  general  topics: 

1 .  Object-oriented  authoring  tools 

2.  Authoring  tools  for  non-IT  personnel 

3.  Cost-effectiveness  tradeoffs 

4.  Authoring  tools  for  specific  capabilities. 

More  specific  R&D  efforts  to  advance  S&T  in  each  of  these  four  general  topics 
include  the  following  suggestions: 

•  Object-Oriented  Authoring  Tools 

-  Extend  the  SCORM  to  integrate  third-party  applications  better,  espe¬ 
cially  those  concerned  with 

—  3-D  displays  and  the  sequencing  of  3-D  content 

—  Multilearner  participation,  collaboration,  and  assessment 
—  Haptics 

—  Psycho-physiological  sensors 
—  Global  positioning  sensors 

—  Speech  recognition  and  natural  language  understanding 

—  Avatars 

-  Design  and  develop  authoring  tools  for  locating  and  accessing  learning 
objects  and  their  repositories 

-  Develop  cross-language  search  capabilities  for  identifying  knowledge 
objects 
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-  Include  assessment  in  object  packages. 

•  Authoring  Tools  for  Non-IT  Personnel 

-  Reduce  the  time  and  resources  needed  to  capture  and  represent 
expertise,  especially  in  complex,  ill-structured  environments 

-  Increase  the  range  of  instructional  strategies,  technologies,  and  sources 
of  information  with  which  authoring  tools  deal 

-  Apply  recent  cognitive  science  achievements  to  help  developers  tailor 
training,  education,  and  diagnostic  assessment  to  individual  learner 
needs 

-  Enhance  automated  capabilities  for  performing  front-end  analysis  and 
identifying  training  objectives,  tasks,  conditions,  and  standards 

-  Design  and  develop  automated  authoring  capabilities  for  rapid  genera¬ 
tion  of  simulations,  scenarios,  and  instructional  materials 

-  Design  and  develop  capabilities  for  generating  instructional  materials 
from  case  studies,  narratives,  and  technical  documentation 

-  Design  and  develop  authoring  tools  for  computer-generated  (automated 
and  semi-automated)  participants  (allies,  enemies,  and  neutrals:  combat¬ 
ants  and  noncombatants) 

•  Cost-Effectiveness  Tradeoffs 

-  Design  and  develop  authoring  tools  that  treat  costs  and  cost-effective¬ 
ness  tradeoffs  seriously,  including  tradeoffs  for 

—  Tailored,  diagnostic  feedback  vs.  simple  re-teaching 

—  Learner  control  vs.  system  control  of  training  content  and 
sequencing 

—  Direct  instruction  vs.  hints 

—  Training  with  and  without  avatars 

—  Training  with  and  without  agents 

—  Game-like  entertainment  vs.  simulation  vs.  direct  instruction 
—  Training  with  and  without  models  of  misconceptions 
—  Training  vs.  performance  aiding 
—  Automated  vs.  human-controlled  participants 
—  Development  with  and  without  the  use  of  sharable  objects 
—  Handoffs  between  individual  and  team/collective  training 
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—  Levels  of  physical  vs.  “psychological”  fidelity 
—  Live  vs.  constructive  vs.  virtual  simulation 

-  Design  and  develop  cost  models  with  clearly  defined  cost  elements  for 
training  and  education  authoring. 

•  Authoring  Tools  for  Specific  Capabilities 

-  Authoring  tools  and  capabilities  to  integrate  training  and  education  with 
perfonnance  aiding,  decision-making,  and  problem  solving 

-  Authoring  tools  and  capabilities  for  producing  assessments  of  student 
progress  linked  to  training  objectives 

-  Authoring  tools  and  capabilities  for  developing  genuinely  entertaining 
games  that  produce  specifically  targeted  training  objectives 

-  Automated  aids  for  designing,  developing,  and  reusing  scenarios 
matched  to  training  objectives 

-  Authoring  tools  and  capabilities  for  case-based  instruction 

-  Authoring  tools  and  capabilities  to  reflect  emotional  states  in  avatars 

-  Authoring  tools  and  capabilities  that  provide  “plug-and-play”  cultural 
modules  for  training  and  assessment  in  multinational  environments 

-  Authoring  tools  and  capabilities  for  agile,  rapid  construction  of  terrain 
databases,  especially  the  urban  terrain  databases 

-  Authoring  tools  and  capabilities  for  performing  training  needs  analyses 

-  Authoring  tools  and  capabilities  for  natural  language  interactions  and 
mixed-initiative  dialogue. 

c.  Simulation-Based  Training 

R&D  needs  in  the  simulation-based  training  area  fall  under  four  general  topics: 

1.  Matching  simulation-based  training  to  training  objectives 

2.  Automated  detection  of  critical  events  and  decisions 

3.  Cost-effective  application  of  simulation-  and  game-based  training 

4.  Development  of  specific  simulation  capabilities. 

More  specific  R&D  efforts  to  advance  S&T  in  each  of  these  four  general  topics 
include  the  following  suggestions: 
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•  Matching  Simulation-Based  Training  to  Training  Objectives 

-  Match  scenarios  and  simulation  fidelity  to  learner  needs  and  training 
objectives 

-  Establish  fidelity  requirements  for  training  and  assessing  “higher  order” 
cognitive  skills 

-  Use  diagnostic  infonnation  from  individuals  and  teams  to  design, 
develop,  and  reuse  scenarios. 

•  Automated  Detection  of  Critical  Events  and  Decisions 

-  Automate  the  assessment  of  audio  communication,  communication  net 
traffic,  and  text  message  traffic 

-  Automate  the  identification  of  critical,  “trigger”  events  and  decisions  for 
training  and  post-simulation  (after  action)  feedback. 

•  Cost-Effective  Application  of  Simulation-  and  Game-Based  Training 

-  Imbue  nonplayer  characters  with  sufficient  knowledge,  skills,  and  abili¬ 
ties  to  ensure  character  believability  without  disrupting  scenarios 

-  Develop  capabilities  to  assess  immersion,  its  effects  on  learning,  and  its 
cost  effectiveness 

-  Identify  optimally  cost-effective  combinations  of  live,  virtual,  and  con¬ 
structive  simulations 

-  Detennine  the  cost  effectiveness  of  games  and  other  highly  motivating 
simulation 

-  Detennine  the  cost  effectiveness  of  noncognitive  effects  and  models 
integrated  into  human  behavior  representation. 

•  Development  of  Specific  Simulation  Capabilities 

-  Simulate  urban  operations  at  night  among  individuals  who  are  using 
different  night  vision  technologies 

-  Incorporate  a  better  understanding  of  interactions  among  different 
behavior  moderators  operating  concurrently 

-  Train  medical  clinical  procedures  using  more  realistic,  responsive, 
haptic  capabilities  with  enhanced  tool-tissue  interactions 

-  Enhance  integration  of  noncognitive  effects  and  models  into  human 
behavior  representation 

-  Develop  a  more  precise,  metrics-based  methodology  for  detennining  the 
transfer  of  surgical  skills  from  simulation  to  the  real  environment 
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-  Enhance  simulation-based  capabilities  for  increasing  SA  of  teams  and 
individuals 

-  Allow  free  and  facile  interchange  between  live  and  virtual  simulation 
participants 

-  Enhance  haptics  and  machine-manikin  interfaces  for  training  combat 
casualty  care. 

d.  Training  System  Design  and  Development 

R&D  needs  in  the  training  system  design  and  development  area  fall  under  three 
general  topics: 

1.  Team  training 

2.  Cognitive  readiness  training  and  assessment 

3.  Specific  training  techniques  (e.g.,  language  and  cultural  training). 

More  specific  R&D  efforts  to  advance  S&T  in  each  of  these  four  general  topics 
include  the  following  suggestions: 

•  Team  Training 

-  Develop  capabilities  to  assess  and  diagnose  shared  understanding  and 
team  processes  from  a  team’s  conduct  of  military  operations 

-  Develop  capabilities  for  rapid  preparation  of  teams — especially  hastily 
assembled  teams — during  pre-deployment  to  accomplish  specific 
missions 

-  Extend  training  to  a  more  comprehensive  range  of  teams — from  crews  to 
command  staffs — with  a  focus  on  communication,  coordination,  and 
knowledge  representation  for  rapidly  formed  “pick-up”  teams 

-  Develop  principles  for  trading  off  individual  training  with  training  that 
must  be  conducted  within  teams. 

•  Cognitive  Readiness  Training  and  Assessment 

-  Account  for  different  types  and  levels  of  cognitive  abilities  when 
training  and  assessing  leaders 

-  Train  leaders  to  recognize  and  respond  to  operational  patterns  in  asym¬ 
metric  environments 

-  Represent  higher  order  cognitive  skills,  such  as  adaptability,  creativity, 
and  critical  thinking 

-  Develop  objective,  quantifiable  measures  for  cognitive  processes  and 
capabilities 
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-  Balance  discrete-outcome  with  continuous-immersion  training  strategies 
in  cognitive  skill  development 

-  Advance  the  scope  of  technology-based  training  and  education  models 
and  applications  beyond  declarative  knowledge  to  those  involving  ana¬ 
lytical  and  evaluative  thinking 

-  Extend  SA  to  a  full  range  of  echelons 

-  Develop  capabilities  for  addressing  the  real-time  analysis  of  tasks  and 
task  workloads 

-  Develop  cognitive  training  and  diagnostic  cognitive  readiness  assess¬ 
ment  for  a  full  range  of  leaders — from  strategic  corporals  to  Combatant 
Commanders 

-  Develop  effective  instruction  in  time-critical  tasks  and  activities,  such  as 
trauma  treatment  and  tactical  decision-making. 

•  Specific  Training  Techniques  (e.g.,  Language  and  Cultural  Training) 

-  Integrate  personnel  records  management  systems  seamlessly  with 
training  and  education  interventions  to  meet  individual  needs 

-  Detennine  the  military  value  of  training  to  assess  the  quantifiable  impact 
of  training  on  military  operation  success 

-  Develop  rapidly  modifiable  medical  training  systems  for  combat 
casualty  care — close  to  “point  of  wounding” 

-  Incorporate  cultural  differences  in  training  for  multinational  operations, 
especially  for  cultural  issues  relating  to  information  sharing  and  deci¬ 
sion-making 

-  Enhance  language  and  culture  preparation  for  coalition  and 
multinational  operations,  within  military  organizations  and  between 
military  and  nonmilitary  participants. 

2.  Prioritization 

How  should  all  these  gaps  be  addressed  and  prioritized  by  DoD  SBIR/STTR 
training  and  education  activity?  One  approach  might  be  to  concentrate  on  the  unique  fea¬ 
tures  of  the  SBIR/STTR  program.  Opportunities  to  create  SBIR/STTR  topics  and  seek 
funding  occur  at  least  once  a  year.  These  “opportunities”  do  not  have  to  be  planned  and 
budgeted  for  several  years  in  advance  and  can  respond  to  promising  technical  approaches 
as  they  arise.  These  opportunities  broaden  the  base  from  which  technical  ideas  can  be 
drawn.  Individual  R&D  program  mangers  have  a  myriad  of  small  business  entrepreneu¬ 
rial  ideas  and  energy  from  which  to  draw.  Flexibility,  rapidity,  and  product  development 
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seem  to  be  hallmarks  of  SBIR/STTR  topics  and  projects.  R&D  activities  where  these 
qualities  are  at  less  of  a  premium  might  be  left  to  other  sources  of  support. 

In  addition,  some  other  areas  are  not  receiving  much  R&D  attention  from  other 
sources  and  might  be  particularly  appropriate  for  SBIR/STTR  funding.  The  following 
(nonexhaustive)  list  includes  some  of  these  topics: 

•  Development  of  practicable  cost-effectiveness  models.  Improved  cost- 
effectiveness  is  frequently  used  as  a  reason  for  R&D  investment.  However, 
cost  models  that  have  well-defined,  comprehensive,  usable,  and  widely 
agreed-upon  cost  elements  are  hard  to  find  in  training  and  education.  These 
models  are  also  rarely  applied  in  assessing  the  military  return  on  investment 
for  training  and  education  R&D  products.  Development  of  cost  models  that 
could  be  linked  to  operational  effectiveness  would  make  the  resource-alloca¬ 
tion  discussions  of  training  and  education  R&D  investments  more  concrete 
and  more  competitive. 

•  Progress  assessment  from  routine  interactions.  Using  the 
communications,  click  streams,  and  keyboard  inputs  recorded  from 
interactive  system  use  to  assess  learner  and  user  knowledge,  skill,  and 
abilities  would  significantly  aid  the  process  of  tailoring  training,  education, 
performance  aiding,  decision  aiding,  information  presentations,  and  so  forth 
to  the  needs  and  capabilities  of  learners,  users,  and  decision-makers. 
Recording  the  information  can  be  done  easily.  Organizing  and  extracting 
useful  information  is  much  more  difficult  and  needs  further  investment  and 
effort.  Modeling  learners  and  users  has  received  considerable  R&D  attention 
for  some  time  (Fletcher,  1975;  Morrison,  2003),  but  the  techniques  for 
extracting  the  needed  infonnation  from  routine  interactions  (aside  from 
explicit  testing)  have  not.  An  SBIR/STTR-supported  effort  to  develop 
techniques  for  this  purpose  would  improve  our  ability  to  create  computer- 
mediated  environments  for  training,  education,  and  perfonnance  aiding  and, 
as  evidenced  by  the  rudimentary  efforts  of  online  shopping  sites,  might  find  a 
commercial  market. 

•  Generation  of  instructional  materials  from  reusable  objects.  Training  and 
education  may  be  evolving  toward  the  use  of  personal  learning  associates 
that,  rather  than  presenting  didactic  lessons,  would  help  learners  attain 
instructional  objectives  by  engaging  them  in  open-ended,  mixed-dialogue 
“conversations”  (Carbonell,  1970;  Chipman,  2003;  Fletcher,  2006;  Graesser, 
Gersbacher,  and  Goldman,  2003).  System  responses  would  have  to  be  assem¬ 
bled  on-demand,  and  in  real  time.  These  responses  would  have  to  be  assem¬ 
bled  from  something,  and  reusable  objects  may  prove  as  good  or  perhaps 
better  for  this  purpose  than  anything  else.  The  development  of  techniques  for 
doing  this  have  not  been  developed  and  are  receiving  limited  R&D  attention. 
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A  capability  for  assembling  tutorial  interactions  on-demand,  in  real  time  (or 
near-real  time),  and  tailored  to  learner  needs  and  abilities  would  advance  the 
state  of  the  art,  reduce  authoring  costs,  and  likely  succeed  in  the  market 
place. 

•  Instructional  techniques  for  enhancing  cognitive  readiness  (agility, 
flexibility,  creativity).  One  aspect  of  military  operations  that  can  most  cer¬ 
tainly  be  expected  is  the  unexpected.  Mission-essential  task  lists  (METLs), 
mission-essential  competencies,  and  mission  rehearsal  are  helpful  in  pre¬ 
paring  individuals  and  units  for  what  can  be  anticipated  in  operations  and 
missions.  However,  some  portion(s)  of  the  mission  will  be  unanticipated, 
unexpected,  and  often  not  prepared  for.  How  can  individuals  and  units  be 
better  prepared  to  recognize  the  unexpected  and  deal  with  it  in  a  creative, 
flexible,  and  agile  manner?  Instructional  techniques  for  improving  the  ability 
to  respond  quickly  to  unexpected  exigencies  in  a  creative,  unplanned  (and 
successful)  manner  could  be  developed.  Such  techniques  would  have  appli¬ 
cation  beyond  military  operations.  These  techniques  are  receiving  only  lim¬ 
ited  R&D  attention  at  present,  but  they  might  find  SBIR/STTR  resources  for 
their  development  and  willing  customers  in  the  marketplace. 

I.  SBIR/STTR  Management 

Some  suggestions  for  managing  the  SBIR/STTR  programs  in  training  and  educa¬ 
tion  arose  from  this  review.  They  concern  (in  no  particular  order)  selecting  SBIR/STTR 
topics  for  training  and  education,  ensuring  innovation,  infrastructure  engineering,  over¬ 
sight  for  commercialization,  evaluating  project  results,  and  submission  of  project  reports. 

1.  Selecting  SBIR/STTR  Topics  for  Training  and  Education 

With  the  notable  exceptions  of  ODUSD(S&T)  Biosystems  and  DARPA,  the  final 
decisions  about  which  topics  are  chosen  and  funded  are  usually  made  with  limited  or  no 
technically  qualified  human  systems  input.  The  number  of  proposals  an  organization  is 
allowed  to  fund  is  more  likely  to  be  detennined  on  the  basis  of  resources  available  than 
on  the  technical  quality  of  proposals  received. 

Benchmarking  for  proposal  scores  differs  between  organizations.  Its  absence  is 
causing  scores  to  be  inflated  to  the  point  that,  in  some  instances,  proposals  that  fail  to 
score  99  or  100  will  not  be  funded.  Management  attention  may  be  needed  to  ensure 

(1)  that  human  systems  expertise  is  included  in  final  selection  of  SBIR/STTR  topics  and 

(2)  that  well-defined  score  benchmarks  are  established  and  applied  when  selecting  pro- 
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posals  for  funding.  The  fonner  may  not  be  difficult  to  provide.  The  latter  will  require 
analysis,  decision-making,  and  exercise  of  leadership. 

2.  Ensuring  Innovation 

The  tension  between  “requirements  pull”  and  “technical  opportunity  push”  is  a 
perennial  and  well-noted  issue  for  DoD  R&D  managers.  It  is  as  real  in  DoD  SBIR/STTR 
training  and  education  as  it  is  elsewhere.  The  task  is  to  seek  intersections  between  the 
two  and  create  commercially  marketable  products  that  are  innovative  in  advancing  the 
state  of  S&T  and  of  practical  military  value  in  advancing  the  state  of  operational  practice. 

The  appeal  of  setting  aside  some  degree  of  innovation  to  produce  a  marketable 
product  for  a  grateful  military  customer  can  be,  for  many  practical  reasons,  irresistible. 
Many  promising  innovations  are  among  the  SBIR/STTR  products  covered  by  this  review, 
but  the  innovation  and  S&T  contribution  evident  in  others  is  marginal.  Achieving  a 
proper  balance  between  innovation  and  short-term  “fixes”  is  always  difficult,  but  the  for¬ 
mer  should  not  be  neglected. 

3.  Infrastructure  Engineering 

Most  innovations  create  a  “ripple  effect”  in  the  technical  environment  around 
them.  Other  activities  may  need  to  adjust  their  products  and  processes,  or  entirely  new 
products  and  processes  may  have  to  be  created  to  enable  the  absorption  and  institutionali¬ 
zation  of  the  innovation  into  its  environment.  The  innovation  itself  may  have  to  be  modi¬ 
fied  or  augmented  to  adjust  to  its  surroundings.  Horseless  carriages  need  filling  stations, 
repair  shops,  and  highways.  Wireless  telegraphs  need  transmitters,  frequencies,  and  a 
radio  advertising  industry.  In  short,  innovations  require  adaptation  to  and  by  the  infra¬ 
structure  into  which  they  are  to  be  introduced. 

Timing  matters.  To  be  introduced  and  adopted  successfully,  an  innovation  needs 
sufficient  supporting  infrastructure  for  its  users  to  realize  its  value  and  potential  contribu¬ 
tions  to  achieving  their  goals.  No  matter  how  technically  advanced  or  promising  an  inno¬ 
vation  may  be,  if  it  is  too  difficult,  complex,  or  simply  awkward  to  use  (i.e.,  if  it  is  intro¬ 
duced  too  soon,  without  a  sufficient  supporting  infrastructure),  it  can  easily  fail  in  the 
marketplace. 

These  considerations  suggest  a  need  for  conscious  and  explicit  attention  to  infra¬ 
structure  development.  It  does  not  seem  unreasonable  to  view  such  development  as  a 
form  of  engineering  that  requires  interdisciplinary  effort  to  ensure  that  the  introduction  of 
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an  innovation  into  its  environment  includes  the  technical,  social,  and  economic  changes 
needed  for  its  successful  adaptation.  Infrastructure  engineering  is  an  essential  aspect  of 
innovation  and  deserves  specific  attention  and  support  in  the  DoD  R&D  enterprise. 

In  the  environment  affected  by  DoD  SBIR/STTR  training  and  education  invest¬ 
ments,  conscious  and  goal-directed  infrastructure  engineering  may  be  needed  to  allow  the 
successful  integration  of  the  innovations  they  produce.  For  instance,  SCORM  may  need 
to  be  harmonized  with  the  HLA  for  simulations  based  on  reusable  objects  to  be 
developed  and  used;  authoring  capabilities  may  need  to  be  modified  or  developed  to 
permit  classroom  instructors  and  students  to  prepare  scenarios  developed  for  haptics- 
based  simulators;  and  classroom  instruction  budgeting  practices  based  on  instructor 
contact  hours  may  need  to  be  adjusted  to  allow  the  promise  of  anytime,  anywhere 
training,  education,  and  perfonnance-aiding  capabilities  to  be  realized  fully. 

In  brief,  conscious  attention  and  resources  may  need  to  be  allocated  to  the  devel¬ 
opment  of  an  infrastructure  engineering  discipline  if  returns  on  SBIR/STTR  investments 
of  all  sorts — including,  of  course,  those  made  in  training  and  education — are  to  be 
obtained  fully  and  successfully. 

4.  Oversight  for  Commercialization 

The  record  for  commercializing  SBIR/STTR  training  and  education  products  is 
spotty.  Phase  3  is  expected  to  proceed  without  further  government  investment.  Once  a 
promising  innovation  is  produced,  the  responsible  firm  is  expected  to  find  other  sources 
of  funding.  All  SBIR/STTR  projects  are  expected  to  develop  a  commercialization  plan 
for  a  marketable  product,  and  many  do  so.  However,  that  may  not  be  enough.  No  guaran¬ 
tees  exist  to  ensure  that  Phase  3  will  be  funded  or  carried  out,  and  significant  SBIR/ 
STTR  innovations  are  often  left  in  the  laboratory. 

The  culture  and  skills  needed  for  R&D  and  technological  innovation  differ  from 
those  required  for  commercialization  and  marketplace  transition.  The  difference  can  and 
evidently  does  handicap  product  development.  A  difficult  challenge  for  many  technically 
inventive  R&D  firms  is  recognizing  and  then  effecting  the  shift  of  resources  (estimated  to 
be  about  50  percent)  from  innovation  to  commercialization  needed  to  carry  out  a  success¬ 
ful  Phase  3  transition.  The  problem  is  mitigated  somewhat  by  Phase  2  extended  funding, 
which  can  help  firms  develop  products  for  use  by  a  specific  agency  (e.g.,  DoD),  but, 
overall,  the  problem  remains.  The  commercialization  track  record  for  SBIR/STTR 
training  and  education  can  be  improved  with  more  oversight  by  government  R&D  project 
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managers  to  ensure  that  firms  entering  Phase  2  incorporate  and  embrace  the  culture  and 
skills  required  for  marketplace  transition. 

5.  Evaluating  Project  Results 

Few  evaluations  of  the  many  SBIR/STTR  products  developed  across  the 
FY  1999-FY  2004  time  period  were  documented  in  project  reports.  Products  that  proceed 
to  commercialization  will,  of  course,  be  subjected  to  marketplace  evaluation  and  thereby 
will  meet  the  SBIR/STTR  program  objective  of  increasing  the  commercialization  of 
innovations  derived  from  federal  R&D.  However,  marketplace  success  depends  on  many 
factors  other  than  the  contribution  that  products  make  to  S&T.  Valuable,  innovative, 
high-quality  products  may  fail  while  others  of  far  less  worth  may  end  up  thriving  in  the 
national  marketplace.  If  we  are  to  “leverage”  and  learn  from  SBIR/STTR  efforts,  more 
attention  (at  least  in  addition  to  marketplace  trials)  should  be  given  to  evaluating  the 
products  and  documenting  their  S&T  value. 

6.  Submission  of  Project  Reports 

Locating  SBIR/STTR  project  reports — particularly  Phase  1  and  Phase  2  Final 
reports — was  often  difficult  and  occasionally  impossible.  Only  57  percent  of  the  Phase  1 
final  reports  that  should  have  been  submitted  to  the  Defense  Technical  Information  Cen¬ 
ter  (DTIC)  could  actually  be  found  there.  The  Services/organizations  differed  in  their  rate 
of  submission:  78  percent  for  Air  Force  projects,  67  percent  for  DARPA  projects, 
60  percent  for  Anny  projects,  52  percent  for  OSD  projects,  and  10  percent  for  Navy 
projects. 

According  to  Department  of  Defense  Directive  (DoDD)  3200.12,  DoD  Scientific 
and  Technical  Information  (STI)  Program  (STIP),  research,  development,  test,  and 
evaluation  (RDT&E)  managers  and  perfonners  are  required  to  use  and  support  the  STIP. 
Further,  performance  of  RDT&E  studies  and  efforts  are  not  considered  complete  until 
their  STI  is  documented  satisfactorily  and  provided  to  applicable  STI  distribution  activi¬ 
ties.  Department  of  Defense  Instruction  (DoDI)  3200.14,  Principles  and  Operational 
Parameters  of  the  DoD  Scientific  and  Technical  Information  Program,  identifies  DTIC 
as  a  central  coordinating  point  for  DoD  STI  databases  and  systems.  It  requires  the  Heads 
of  DoD  components  to  ensure  that  S&T  findings  and  results  derived  from  DoD  contracts 
and  grants  are  recorded  as  technical  documents.  They  are  further  required  to  make  these 
technical  documents  available  to  the  DoD  community  through  DTIC  and  other  collection 
and  distribution  activities  that  seem  appropriate.  Technical  documents  are  defined  by 
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DoDI  3200.14  as  “any  recorded  information  that  conveys  STI  or  technical  data, 
regardless  of  media.”  Notably,  STI  documentation  in  the  form  of  journal  articles  and 
poster  papers  at  symposia  must  also  be  provided  to  DTIC. 

The  Defense  Federal  Acquisition  Regulations  Supplement  (DFARS)  252.235  of 
November  2004  also  states  that  DTIC  is  responsible  for  collecting  all  scientific  and  tech¬ 
nical  reports  for  DoD  (DFARS  235.010)  and  that  R&D  contractors  shall  submit  two  cop¬ 
ies  of  approved  scientific  or  technical  reports  to  DTIC. 

It  therefore  appears  to  be  a  DoD  requirement  that  SBIR/STTR  reports  be  submit¬ 
ted  to  DTIC.  If  nothing  else,  such  a  requirement  would  help  to  ensure  that  SBIR/  STTR 
STIP,  in  the  words  of  DoDD  3200.12,  “materially  impacts  DoD  ability  to  leverage  sig¬ 
nificant  investments  in  defense  technology.”  Requirement  or  not,  many  SBIR/STTR  pro¬ 
jects  develop  truly  innovative  products,  procedures,  and  technologies.  Their  findings  can 
contribute  significantly  to  S&T  and  should  be  shared  with  the  DoD  R&D  community  and 
beyond. 

Some  SBIR/STTR  reports  may  contain  proprietary  infonnation  needed  for  their 
commercial  success.  DTIC  has  adequate  provisions  for  protecting  these  reports.  Anecdo¬ 
tal  evidence  in  SBIR/STTR  training  and  education  suggests  that  providing  these  protec¬ 
tions  is,  at  most,  a  minor  problem.  In  brief,  more  DoD  SBIR/STTR  reports  from  training 
and  education  projects  should  find  their  way  at  least  to  DTIC. 

J.  A  Final  Word 

The  creativity  and  energy  of  participating  training  and  education  R&D  managers 
combined  with  the  productivity  and  inventiveness  of  the  small  R&D  firms  that  specialize 
in  human  systems  attest  well  to  the  significant  value  of  the  DoD  SBIR/STTR  program  in 
advancing  both  S&T  and  DoD  operational  capabilities.  The  program  seems  to  be  doing 
for  training  and  education  precisely  what  its  authorizing  legislation  intended.  Manage¬ 
ment  issues  concerning  reports,  commercialization,  innovation,  evaluation,  and  topic 
selection  require  some  attention,  but  these  issues  are  by  no  means  unresolvable. 

In  short,  much  about  the  DoD  SBIR/STTR  training  and  education  activity  is 
commendable.  R&D  managers  and  several  firms  have  taken  advantage  of  this 
opportunity  and  should  be  commended.  Participation  in  the  program  should  be  continued 
and  encouraged — if  not  expanded. 
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Note:  An  asterisk  (*)  (e.g.,  *OSD)  indicates  that  the  topic  was  added  after  the  June  2005 
Workshop. 

Note:  A  =  Army;  N  =  Navy;  AF  =  Air  Force;  D  =  Defense  Advanced  Research  Projects 
Agency  (DARPA);  OSD  =  The  Office  of  the  Secretary  of  Defense. 


Intelligent  Tutoring  Systems  (ITSs) 


AF  2002-085 

Adaptive  Training  for  Real-time  Intelligence  Monitoring  and  Evaluation 

Objective:  Develop  an  innovative  capability  to  support  information  operations 
and  information  warfare  training  using  cognitive  modeling,  knowledge  engi¬ 
neering,  and  formal  representations  of  knowledge. 

A  1999-T006 

Creating  Knowledge  and  Improving  Training  Through  Latent  Semantic  Indexing 
(STTR) 

Objective:  Develop  and  evaluate  novel  ways  for  creating  face-to-face  interaction, 
coaching,  and  feedback  and  for  providing  rapid  access  to  expertise. 

A  2000-010 

Automatic  Adaptive  Support  for  Selection  and  Rapid  Team  Building  Leadership 
Skills  Using  Latent  Semantic  Analysis 

Objective:  Improve  the  capabilities  of  Latent  Semantic  Analysis  (LSA)  to 
provide  (especially  in  Distance  Learning)  instant  individualized  feedback  to 
instructors  and  students  on  written  essays  and  memos. 

A  2003-204 

Adapting  Intelligent  Tutoring  System  for  Assessing  Collaborative  Skills 

Objective:  Develop  an  automated  event  recognition  capability  to  support  per¬ 
formance  assessment  for  simulation-based  teamwork  training  in  massively  multi¬ 
player  (MMP)  gaming  environments  for  training  small  dismounted  infantry  teams 
in  simulated  urban  environments. 

A  2003-T001 

The  Virtual  Observer /Controller  (O/C) — Intelligent  Coaching  in  Dismounted 
Warrior  Simulations  (STTR) 

Objective:  Develop  an  automated  observer/controller  for  training  in  virtual  envi¬ 
ronments  that  can  assess  soldier  performance  compared  with  standards. 
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D  2002-030 

Real-time  Assessment  of  Student  State 

Objective:  Develop  the  capability  to  build  in  real  time  a  multidimensional  model 
(including  physiological,  psychological,  and  knowledge  information)  of  the  stu¬ 
dent  and  change  teaching  behavior  based  on  this  model. 

OSD  2000-CR01 

Automated  Dialogue  Modeling  Using  Natural  Language  Understanding  in  ADL 
by  the  Naval  Air  Warfare  Center  Training  Systems  Division 

Objective:  Develop  a  tutoring  application  that  can  be  used  in  distributed  training 
to  respond  with  natural  language  output  or  topic  selection  to  natural  language 
queries  from  the  student  about  the  subject  area. 

OSD  2000-CR02 

Training  Users  ’  Cognitive  Readiness  for  Combat  Command  Using  an  Intelligent 
Tutor  To  Model  Expert  Mentor  Interactions  by  the  Army  Research  Institute  for 
the  Behavioral  and  Social  Sciences  (ART) 

Objective:  Develop  automated  tutoring  for  an  open-ended  domain  and  provide 
training  for  more  adaptive  command  leadership. 

OSD  2000-CR07 

Personal  Education  and  Training  Assistant  (PETA)  for  Distance  Learning  by  the 
Naval  Air  Warfare  Center  Training  Systems  Division 

Objective:  Develop  an  embodied,  intelligent  interface  agent  that  provides  the 
learner  with  counseling  and  mentoring;  pedagogical  support;  course  selection  and 
scheduling;  testing  and  perfonnance  feedback;  learning  management  services; 
and  career  management  services. 

OSD  2001-CR12 

The  Grain  Size  of  Student  Models  as  a  Factor  in  ICAI  Effectiveness  by  the  Office 
of  Naval  Research 

Objective:  Determine  whether  the  grain  size  of  the  student  model  is  a  significant 
factor  in  training  effectiveness. 

*OSD  2002-DH10 

Generative,  Knowledge-based  Approaches  for  Rapid  Development  of  Simulation- 
based  Medical  Training,  Air  Force  Research  Laboratory 

Objective:  Develop  a  generative  knowledge-based  instructional  system  to  rapidly 
produce  simulation-based  training  for  medical  procedures. 


A-4 


General  Authoring  Tools 


AF  2000-100 

Force  Protection  Training  Technology 

Objective:  Design  and  implement  tools  for  training  development  modularization 
and  software  delivery;  distributing  just-in-time  (JIT)  training  content  during  mis¬ 
sion  preparation;  and  cross-training  deployable  environments  and  civilian 
domain. 

A  2002-024 

Embedded  Training  for  Objective  Force  Warrior 

Objective:  develop  and  evaluate  prototype  instructional  design  methodology, 
authoring  tools,  and  training  support  for  embedded  training  with  projected  Objec¬ 
tive  Force  Warrior  (OFW)  technology  and  functions. 

N  2001-011 

Web-based  and  Traditional  Classroom  Lesson  Design  Guide 

Objective:  Develop  a  computer-based  instructional  design  tool  that  presents  heu¬ 
ristics  and  guidance  for  designing  effective  training  for  traditional  (classroom) 
and  Web-based  environments. 

OSD  2000-CR04 

Digital  Resource  for  Instructional  Design  in  CBT  Authoring  Environments  by  the 
Office  of  Naval  Research  (ONR) 

Objective:  Develop  a  design  tool  for  courseware  development  tool  to  assist  non¬ 
expert  courseware  developers,  including  subject  matter  experts  (SMEs)  and  oth¬ 
ers,  in  using  expert  instructional  design  methods. 

OSD  2000-CR05 

Dismounted  C4ISR  Data  Presentation  and  Dissemination  by  the  Army  Research 
Laboratory  (ARL) 

Objective:  Create  a  toolkit-based  development  environment  for  creating  military 
training  applications  using  a  multitargeted  (game  architecture)  approach;  Exploit 
Geographic  Infonnation  System  (GIS)  collections  to  create  geo-realistic  game 
playing  environments  using  automated  feature  recognition  and  extraction. 
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3-D  Components  for  Virtual  Environments  by  the  Army  Simulation  and  Training 
Command  (STRICOM) 

Objective:  Develop  a  framework  and  supporting  authoring  tools  for  the 
structured  design  of  3-D  content  based  on  reusable  components  conformant  with 
the  Sharable  Content  Object  Reference  Model  (SCORM). 

OSD  2001-CR11 

Authoring  Shell  for  Case-Based  Instruction  by  the  Office  of  Naval  Research 

Objective:  Provide  authoring  tools  for  case-based  instruction  that  reflect  recent 
results  from  ONR-sponsored  research  on  effective  case-based  instruction. 

*OSD  2001-CR13 

Toolbox/Intelligent  Advisor  for  Creating  Pedagogically  Correct,  In  teresting,  and 
Motivating  Instructional  Content  by  the  Naval  Air  Warfare  Center 

Objective:  Enhance  Advanced  Distributed  Learning  (ADL)  on  the  Web  through 
intelligent  design  and  authoring  software  based  on  pedagogical  issues,  principles, 
and  standards  while  focusing  on  learning  object  definitions  that  reap  a  return  on 
investment. 

OSD  2001-CR14 

Intelligent  Assistant  for  Web-based  Training  Vignette  Design  by  the  Naval  Air 
Warfare  Center 

Objective:  Develop  for  short  sequences  of  instruction  a  content  creation  tool  that 
focuses  on  problem  solving,  decision-making,  and  high-level  cognition,  is  based 
upon  a  single  instructional  objective,  is  developed  around  a  training  scenario,  and 
supports  content  management,  distribution,  and  SCORM  specification. 

*OSD  2002-CR13 

Design  of  Sharable  Conten  t  Objects  With  Return  on  Investment  by  the  Naval  Air 
Warfare  Center 

Objective:  Enhance  ADL  on  the  Web  through  intelligent  design  and  authoring 
software  based  on  pedagogical  issues,  principles,  and  standards  while  focusing  on 
learning  object  definitions  using  SCORM  with  extended,  interoperable,  and  dis¬ 
coverable  meta-data  tags. 
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Simulation  Authoring  Tools 


AF  2002-068 

DMT  Training  Requirements  and  Capability  Analysis 

Objective:  Streamline  and  optimize  the  training  systems  requirements  analysis 
process  and  associated  database  to  support  development  of  federations  for  Dis¬ 
tributed  Mission  Operations  (DMO)  training. 

AF  2002-070 

Time  Critical  Targeting  Cell  (TCTC)  for  Team  Training  and  Evaluation 

Objective:  Design  and  develop  a  synthetic  team  task  environment  representing  a 
TCTC  for  training  and  evaluation  of  command  and  control  (C2)  strategies  and 
operations. 

AF  2003-052 

In  telligen  t  Scenario  Generation  Tools  for  Training  and  Rehearsal 

Objective:  Develop  software  tools  to  generate  training  automatically  scenarios 
that  meet  syllabus  requirements  and  target  the  development  of  student  skills  and 
competencies  identified  as  weak,  deficient,  or  in  need  of  refresher  training. 

*AF  2003-058 

Simulation  Models  for  Satellites 

Objective:  Develop  a  specific  simulation  technology  with  rotation  and  zoom 
capabilities  to  allow  visuals  of  internal  equipment  and  equipment  bays  for 
spacecraft. 

A  2002-027 

Training  Rapid  Decision-making  Processes  Required  by  the  Dismounted  Objec¬ 
tive  Force  Leader 

Objective:  Develop  interactive  computer-based  methods  to  train  the  rapid  deci¬ 
sion-making  processes  needed  by  dismounted  leaders  in  both  conventional  and 
digital  battlefield  environments. 

N  2002-184 

Training  Simulation  Intelligent  Scenario  Generation  Tools 

Objective:  Develop  tools  with  distributed  intelligent  support  that  reduce  the  cog¬ 
nitive  workload  for  instructors,  enable  efficient  scenario  planning,  improve 
training  effectiveness  of  limited  simulator  resources,  and  enhance  training 
outcomes. 
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*OSD  2001-CR08 


Tactics,  Training,  and  Procedures  for  the  Warfighter  Reacting  to  Crowd  Dynam¬ 
ics  by  the  Air  Force  Research  Lab  Human  Effectiveness  Directorate,  Brooks  AFB 

Objective:  Develop  tactics,  training,  and  procedures  for  users  of  lethal  and  non- 
lethal  weapons  (NLWs)  that  enhance  the  users’  ability  to  assess  the  situation  rap¬ 
idly  and  make  better  real-time  decisions  that  optimize  the  force  required  to  meet 
mission  objectives  while  minimizing  conflict  escalation. 

*OSD  2001-CR09 

Cognitive  Demands  of  Warfighter  Readiness  by  the  Air  Force  Research  Lab 
Human  Effectiveness  Directorate,  Williams  AFB 

Objective:  Identify  the  cognitive  skills  that  contribute  most  heavily  to  effective 
combat  mission  planning  for  aviators,  develop  DMO  training  interventions 
designed  to  develop  these  cognitive  skills,  and  evaluate  the  impacts  on  subsequent 
mission  planning  performance. 

OSD  2002-CR14 

Multimodal  Visualization  for  Virtual  Environments  Training  Systems  by  the 
Naval  Air  Warfare  Center 

Objective:  Develop  a  design  tool  that  provides  guidance  to  command,  control, 
communications,  computers,  intelligence,  surveillance,  and  reconnaissance 
(C4ISR)  system  designers  regarding  multimodal  infonnation  presentation  and 
develop  a  manning  assessment  tool  to  identify  operators  who  have  a  high  capacity 
for  monitoring  complex,  multimodal,  information-rich  environments. 


Computer-Generated  Participants’  Authoring  Tools 


AF  2000-098 

Enhancing  the  Usability  of  Computer-Generated  Forces 

Objective:  Develop  and  demonstrate  computer-generated  forces  that  are  easier  to 
build,  understand,  deploy,  and  control  during  exercises. 

A  2001-027 

Virtual  Simulation  Tools  for  Cultural  Familiarization 

Objective:  Develop  and  deliver  experiential  training  of  cross-cultural  skills 
within  a  mission  context  and  develop  new  technology  to  train  soldiers  to  under¬ 
stand  cultural  differences  and  to  develop  appropriate  interaction  strategies  in  a 
mission  context  using  highly  experiential,  scenario-based  training  in  virtual 
environments. 
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Simulation  Training  for  Teams 


AF  1999-096 

Distributed  Team  Knowledge  Representation  and  Scenario-Based  Performance 
Evaluation  Methods 

Objective:  Measure  performance  by  developing  efficient,  reliable,  and  valid 
methods  to  represent  knowledge  and  to  decompose  combat  training  scenarios 
rapidly  to  identify  critical  measurement  points  and  performance  criteria. 

AF  2000-080 

Agent-based  Measurement  System  for  Advanced  Distributed  Learning 
Technologies 

Objective:  Develop  predictive  models  of  complex  performance  to  pennit  diagno¬ 
sis  and  prescription  in  distributed  (learning)  environments  such  as  DMO  training. 

AF  2002-071 

Distributed  Interactive  Training  for  the  C2  Aerospace  Operations  Center  (AOC) 

Objective:  Define  cognitive  models  or  framework  to  depict  the  horizontal  and 
vertical  integrated  functions  within  and  throughout  the  AOC,  design  instructional 
strategies  to  teach  the  integrated  functions,  and  develop  preliminary  system  archi¬ 
tectural  specifications. 

AF  2002-078 

Messaging  Interaction  Simulation 

Objective:  Enhance  space  operations  training  by  increasing  the  fidelity  of  space¬ 
craft  simulation  training  environments  by  adding  critical  Space  Operations  Center 
support  applications  and  providing  instructors  and  trainees  with  training  and 
evaluation  support  tools. 

*AF  2004-070 

Distributed  Planning,  Debriefing,  and  After-Action  Review  Capability 

Objective:  Develop,  demonstrate,  and  evaluate  a  distributed  planning,  debriefing, 
and  after-action  review  (AAR)  capability  for  long-haul,  distributed,  computer- 
driven  simulations  in  support  of  team  coordination  training. 

A  2000-097 

Assessing  Decision-making  Skills  in  Virtual  Environments 

Objective:  Develop  a  method  for  automated  assessment  of  small-unit  leader  deci¬ 
sion-making  skills  while  operating  in  virtual  environments. 
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A  2003-031 

Advancing  the  Objective  Force  Through  Multinational  Coalitions  and  Inter¬ 
agency  Task  Forces 

Objective:  Link  with  emerging  technologies  designed  to  improve  the  Army’s 
collaborative  infonnation  environment  and  facilitate  execution  of  Effects  Based 
Operations  (EBOs). 

A  2003-200 

Advanced  Virtual  Environment  Haptic  Simulation 

Objective:  Review  potential  solutions  for  stimulating  the  sense  of  touch  in  a  vir¬ 
tual  environment,  developing  a  prototype  using  full-body  haptic  technologies,  and 
integrating  it  into  the  Army’s  virtual  simulation  environment. 

OSD  2003-DH04 

Deployable  Simulation  Training  for  Operational  Medical  Personnel  &  Emer¬ 
gency  Responders  (OSD/DHP)  by  the  AFRL  Wright  Patterson  Air  Force  Base 

Objective:  Accelerate  the  acquisition  of  expertise  in  decision-making  and  team 
coordination  by  developing  an  intelligent  simulation  training  environment  to  sup¬ 
port  operational  medical  professionals  on  station  and  in  a  deployed  status. 

Simulation  Training  for  Individuals 

AF  1999-082 

Training  for  Space  Operators  Using  a  Distributed  Mission  Training  (DMT) 
Environment 

Objective:  Develop  mission-ready  space  operators  through  DMT  integration  and 
realistic  pre-combat  rehearsal  exercises  for  space  teams. 

*AF  2001-053 

A  Software  Agent  Advisor  for  Satellite  Command  Composition  and  Training 

Objective:  Develop  a  software  agent  for  satellite  ground  systems  operators  to 
support  the  training  and  measurement  of  satellite  command  composition  and 
operations. 

*AF  2001-054 

A  Decision  Aid  for  a  Surveillance  Satellite  Crew  Shift  Supervisor 

Objective:  Develop  a  measurably  effective  decision  aid  for  a  surveillance 
satellite  crew  shift  supervisor. 
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*AF  2001-055 

A  Satellite  Pre-Pass  Contact  Support  Aid 

Objective:  Develop  a  software  agent  for  satellite  pre-pass  contact  operations  and 
support  training. 

AF  2003-045 

Personal  Computer  (PC)-Based  Aircraft  Training  System  and  Visualization  Tool 

Objective:  Develop  a  PC-based  system  to  train  landing  patterns  and  radio  proce¬ 
dures  during  traffic  pattern  operations. 

AF  2003-053 

Time  Critical  Targeting  Training  and  Rehearsal  Environment 

Objective:  Design  a  realistic,  fast-paced,  interactive,  scenario-based  system  to 
train  personnel  on  the  Time  Sensitive  Targeting  (TST)  process  and  develop  and 
hone  the  cognitive  skills  needed  to  apply  the  process  across  a  wide  spectrum  of 
situations. 

AF  2003-064 

Simulation  and  Training  Development  To  Enhance  the  Tactical  Knowledge  and 
Readiness  of  Information  Warfare  Teams 

Objective:  Develop  a  training  capability  to  facilitate  the  acquisition  of  knowledge 
and  skills  associated  with  social  engineering  (e.g.,  train  students  to  play  the  role 
of  a  civilian  computer  technician  in  order  to  infiltrate  an  organization  and  gain 
access  to  an  objective  computer  network). 

*AF  2004-069 

Enhancing  Commanders  ’  Cognitive  Readiness  at  the  Operational  Level  of  War 

Objective:  Develop  scenario-based  training  capability  for  operational-level 
leaders  using  advanced  archiving/retrieval  of  contextualized  developmental 
experiences. 

A  2000-098 

Training  Media  to  Support  Night  Operations  in  Urban  Settings 

Objective:  Develop  computer-based  technologies  to  improve  soldiers’  knowledge 
of  night  equipment  and  transfer  of  skills  to  urban  settings  and  provide  simulated 
cross-training  experience  to  improve  employment  of  night  equipment  in  small- 
unit  operations. 
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A  2002-055 

Software-Driven  Virtual  Minefield 

Objective:  Develop  realistic  training  with  real-time  feedback  that  combines  a 
virtual  minefield  with  computer-modeled  sensor  signals  corresponding  to  physical 
target  signatures  in  various  soil  and  environmental  conditions. 

A  2001-180 

Needle  Thoracentesis  Simulation  Workstation  for  Medical  Training 

Objective:  Create  “virtual  patients”  so  physicians  can  acquire  and  maintain  skills 
needed  to  perform  needle  thoracentesis  procedures. 

A  2001-T007 

Telemedicine  and  Advanced  Medical  Technology  -  Medical/Surgical,  Mission 
Support  Modeling,  and  Simulation  (STTR) 

Objective:  Create  “virtual  patients”  so  physicians  can  acquire  and  maintain  skills 
needed  to  perform  central  venous  catheterization. 

A  2002-184 

Medical  Modeling  &  Simulation — Exsanguinating  Hemorrhage  From  Limbs 

Objective:  Develop  simulation-based  training  to  teach  all  potential  users,  com¬ 
batants,  and  medical  personnel  how  to  apply  a  tourniquet  to  stop  acute  exsan¬ 
guinating  hemorrhage  from  limbs. 

A  2002-T017 

Telemedicine  and  Advanced  Medical  Technology — Refined  Training  Tools  for 
Medical  Readiness  (STTR) 

Objective:  Develop  and  apply  new  tools  to  evaluate  and  report  the  effectiveness 
of  simulators  and  simulation  systems  for  training  medical  personnel  in  key 
combat  casualty  care  skills  by  exploiting  and/or  modifying  systems  used  in 
military  aviation  and  aerospace  industries. 

A  2003-162 

Haptics-Optional  Surgical  Training  System  (HOSTS) 

Objective:  Develop  and  demonstrate  a  computer-based  HOSTS  for  training  open 
surgery  procedures. 
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OSD  2002-CR03 


Dismounted  Infantry  Situational  Awareness  Assessment  in  Virtual  Simulations  by 
the  Army  Research  Institute 

Objective:  Develop  an  unobtrusive  system  for  measuring  situation  awareness 
(SA)  at  squad  level,  rapidly  tailoring  content  and  format  for  AAR  performance 
feedback,  and  providing  in-depth  measures  that  support  research  and  training 
management. 

OSD  2002-DH03 

Medical  Modeling  &  Simulation — Advanced  Ureteroscopy  Simulation  Work¬ 
station  for  Medical  Training  by  the  Army  Medical  Research  Acquisition  Activity 
TATRC 

Objective:  Develop  a  proof-of-concept  and  design,  develop,  build,  and  demon¬ 
strate  a  PC-based  ureteroscopic  endoscopic  surgical  simulation  training  system 
for  training  military  and  civilian  health  care  professionals. 


Training  Systems  Design  and  Development 


AF  1999-083 

Modeling  and  Simulation  of Less-Tlian-War  Scenarios 

Objective:  Develop  innovative  training  strategies  and  approaches  and  evaluate 
their  efficacy  for  Operations  Other  Than  War  (OOTW)  training. 

AF  2002-072 

Integrated  Satellite  Operations  Training  and  Rehearsal  for  Multiple  Satellite 
System  Ground  Control 

Objective:  Develop  a  high-fidelity  integrated  operator  training  and  rehearsal 
environment  for  multiple  satellite  system  control. 

AF  2002-263 

e-Learning  and  Aptitude  Evaluation  through  a  Web-based  Training  Framework 

Objective:  Create  an  advanced  SCORM  compliant  open  system  for  Just-In-Time 
eTraining,  eTesting  (JITCube),  and  Web-based  e-Learning. 

A  1999-011 

Development  and  Test  of  a  Framework  for  Critical  Thinking  Skills  in  a  Military 
Context 

Objective:  Develop  and  evaluate  training  for  critical  thinking  skills  (CTSs)  by 
developing  a  conceptual  model  that  integrates  CTS,  knowledge,  attitudes,  other 
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relevant  cognitive  variables,  battle  command  (BC)  tasks,  and  perfonnance  within 
a  BC  context. 


A  2002-026 

Planning  Exercise  System  To  Promote  Shared  Mental  Models 

Objective:  Improve  the  efficiency,  effectiveness,  and  innovativeness  of  the  com¬ 
mand  group  planning  process  by  developing  and  evaluating  a  training  system  that 
relies  on  case-based  group  training  for  promoting  shared  understanding. 

A  2002-T001 

Leader  Self-development  Support  Program  (STTR) 

Objective:  Promote  effective  self-development  for  leaders  by  defining,  investi¬ 
gating,  and  developing  capabilities  for  self-appraisal,  self  regulation,  and  self¬ 
development. 

N  2001-101 

Maintenance  Skills  Training  through  Distributed  Learning  Principles 

Objective:  Design,  test,  and  field  a  distributed  learning  system  that  supports  the 
development  of  core  and  advanced  technical  skills  through  Web-enabled  interac¬ 
tive  mentoring  and  technology-infused  curricula  that  provide  the  physical  stimu¬ 
lation  of  the  senses  found  in  “stand-up”  classroom  instruction. 

N  2001-116 

Embedded  Training  in  an  Optimized  Manning  Environment 

Objective:  Research,  design,  and  develop  a  prototype  training  system  employing 
closed-loop  adaptive  training  technology  as  the  enabling  technology  for  sup¬ 
porting  training  in  an  optimized  manning  environment. 

N  2002-034 

Scalability  and  Reusability  Methods  for  Intelligent  Tutors  and  Job  Performance 
Aids  for  the  Maintenance  of  Reduced  Manning  Ships 

Objective:  Provide  access  to  real-time,  contextual  knowledge,  either  locally  or 
through  distance  support,  for  maintaining  equipment  on  reduced  manning  ships. 

OSD  1999-004 

Adaptive  Instructional  Systems  by  the  U.S.  Army  Research  Institute  (ARI),  Ft. 
Rucker 

Objective:  Develop  an  approach  for  the  design  and  implementation  of  CBT  sys¬ 
tems  that  dynamically  adapt  instructional  methodology  to  individual  differences 
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in  learning  style  and  rate,  capitalize  on  student  strengths,  and  match  content  and 
structure  of  training  events  to  the  student’s  conceptual  structure. 


OSD  2000-CR06 

Enhancing  Situation  Awareness  in  Military  Operations  by  the  ARI Benning 

Objective:  Develop  and  validate  methods  for  enhancing  the  individual  SA  of 
small-unit  leaders  in  military  operations. 

*OSD  2000-CR09 

Cognitive  Learning  Strategies  for  Medical  Skills  Training  and  Sustainment  via 
Distance  Learning  Means  by  the  Special  Operations  Command 

Objective:  Identify  the  cognitive  skills  that  contribute  most  heavily  to  effective 
combat  mission  planning  for  aviators,  develop  training  interventions  designed  to 
develop  these  cognitive  skills,  and  evaluate  the  impacts  on  subsequent  mission 
planning  performance. 

OSD  2001-CR06 

Scenario-based  Decision  Skills  Training  for  Geographically  Distributed  Teams 
by  the  Air  Force  Research  Lab  Human  Effectiveness  Directorate 

Objective:  Develop,  implement,  and  evaluate  a  scenario-based,  distributed 
training  system  to  enhance  team  decision-making  skills. 

OSD  2001-CR07 

Professional  Leadership  Developmen  t  Skills  Training  for  the  21st  Cen  tury  by  the 
Air  Force  Research  Lab  Human  Effectiveness  Directorate 

Objective:  Develop  a  computer-based,  Internet-accessible  leader  training  system 
that  targets  junior  Air  Force  officers  with  limited  leadership  experience  in  the 
interpersonal  skills  needed  for  unit  leadership. 

OSD  2001-CR10 

Assessmen  t  Methods  for  Tactical  Knowledge  and  Cognitive  Readiness  of  Intelli¬ 
gence  Tasking,  Processing,  Exploitation,  and  Dissemination  (TPED)  Teams  by 
the  Williams  AFB 

Objective:  Develop  automated  methods  to  assess  tactical  knowledge  and  cogni¬ 
tive  readiness  of  Intelligence  TPED  teams. 
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*OSD  2001-CR15 

Instructional  System  for  Enhancing  Sea-keeping  Cognitive  Readiness  and  Deci¬ 
sion-making  Skills  by  the  Special  Operations  Command 

Objective:  Develop  a  training  system  that  enhances  safety  by  improving  operator 
sea-keeping  readiness  and  decision-making  skills. 

OSD  2002-CR01 

Warrior  Readiness  for  Coalition  and  Collaborative  Teams 

Objective:  Design,  implement,  and  evaluate  a  training  and  assessment  tool  for 
leaders  of  multinational/multicultural  teams. 

OSD  2002-CR06 

Integration  of  Behavior  Moderators  into  Cognitive  Performance  Models  for 
Assessing  Cognitive  Readiness,  Natick  Soldier  Center,  in  Collaboration  With 
Army  Medical  Research  and  Material  Command  (MRMC) 

Objective:  Develop  a  comprehensive  model  of  behavior  moderators  present  in 
combat  environments  and  their  effects  on  the  individual  combatant’s  ability  to 
accomplish  cognitive  tasks  and  develop  a  full-spectrum  model  of  cognitive  func¬ 
tioning  applicable  to  combat  environments  in  which  behavioral  moderators 
attenuate  or  prevent  cognitive  deficits. 

OSD  2002-CR15 

Web-based  Game  Design  Advisor,  Naval  Air  Warfare  Center  in  Collaboration 
With  the  Army  Research  Institute 

Objective:  Develop  a  set  of  game  design  recommendations  for  Distance  Learning 
courses  in  the  form  of  a  Web-based  design  aid. 

OSD  2002-DH06 

Computer  Based  Simulation  Technology  for  Training  Technical  Skills  in 
Medicine  by  the  Army  Medical  Research  Acquisition  Activity  (USARIEM) 

Objective:  Design  and  develop  unobtrusive  techniques  and  methodologies  to  pre¬ 
dict  individual  and  team  readiness  to  perform  critical  medical  mission  objectives. 

OSD  2002-DH09 

Global  Treatment  Protocol  Course  via  Advanced  Distributive  Learning  by  the  Air 
Force  Research  Laboratory 

Objective:  Standardize  common  patient  assessment/treatment  protocols,  develop 
ADL/SCORM-compliant,  sustainment-level,  Internet-accessible  training  with 
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downloadable  job  aids  for  using  these  protocols  and  create  an  infrastructure  for 
measuring  and  tracking  first-responder  readiness. 


OSD  2003-DH07 

Distributed  Medical  Training  for  Force  Mobilization  and  Disaster  Relief  by  the 
Office  of  Naval  Research 

Objective:  Develop  and  assess  computer-based  capabilities  that  can  be  used  in  a 
variety  of  environments  (e.g.,  on  the  Web,  in  transit,  in  the  field)  for  rapid 
retraining  and/or  refresher  training  in  the  medical  procedures  needed  for  force 
mobilization  and  disaster  relief. 
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Appendix  B. 

2005  Small  Business  Innovation  Research/ 
Small  Business  Technology  Transfer  (SBIR/STTR) 
Workshop  Slide  Templates 
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•  SBIR/STTR  Topic  Number  and  Title.  For  example,  AF2003-045,  Personal  Com¬ 
puter  (PC)-Based  Aircraft  Training  System  and  Visualization  Tool. 

•  Distribution  Statement.  As  shown. 

•  Briefer  information.  Self-explanatory. 
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/  Place 

SBIR/STTR  Topic  Number  &  Title  (  $  ) 

\  here  J 


Place 
lab  or 


Cluster 


Problem/lssue 


Objective 


The  purpose  of  this  slide  is  to  give  the  audience  a  quick  understanding  of  the  topic. 

•  Heading.  Fill  in  with  correct  information  (see  previous  page). 

•  Cluster.  Please  use  the  cluster  title  from  the  agenda  where  this  SBIR/STTR  topic 
will  be  briefed,  as  follows: 

-  Monday  p.m.  -  Authoring 

-  Tuesday  a.m.  -  Simulation  Training  for  Teams 

-  Tuesday  p.m.  -  Instructional  System  Development 

-  Wednesday  a.m.  -  Intelligent  Tutoring  Systems 

-  Thursday  a.m.  -  Simulation  Authoring 

-  Thursday  p.m.  -  Computer-generated  Technologies. 

USE  SHORT  BULLETS  TO  ANSWER  THE  FOLLOWING 

•  Problem/lssue.  What  problem  is/was  this  topic  intended  to  solve  or  what  issue 
is/was  it  intended  to  address? 

•  Objective(s).  What  were/are  the  broad  goals  of  this  SBIR/STTR  research  topic? 
Where  possible,  quantify. 
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E 


SBIR/STTR  Topic  Number  &  Title 


Phase  1 

Phase  II? 

Title  S  only  if  different  from  header 

Performer 

Objective 

Results/Product 

Title  S  only  if  different  from  header 

Performer 

Objective 

Results/Product 

Title  S  only  if  different  from  header 

Performer 

Objective 

Results/Product 

The  purpose  of  this  slide  is  to  list  and  describe,  as  briefly  as  possible,  the  Phase  I  and 
Phase  II  projects  that  were  funded  under  this  SBIR/STTR  topic.  It  is  also  intended  to  show  which 
Phase  I  projects  were  extended  into  Phase  II. 

•  Heading.  As  in  slide  2. 

For  each  Phase  I  SBIR/STTR  in  the  topic,  please  give: 

•  Title.  The  SBIR/STTR  contract  title  (can  be  shortened  to  conserve  space).  For 
example,  Personal  Computer  (PC)-Based  Aircraft  Training  System  and  Visualiza¬ 
tion  Tool  could  be  rewritten  PC-based  Aircraft  Training  and  Visualization.  You  do 
not  need  to  restate  the  SBIR/STTR  contract  title  if  is  the  same  as  the  topic  title. 

•  Performer.  The  name  of  the  SBIR/STTR  contractor  company  and  principal  investi¬ 
gator  (PI). 

•  Objective.  A  short  objective  for  the  Phase  I  [e.g.,  a  state-of-the-art  analysis  of  stu¬ 
dent  models  in  artificial  intelligence  (AI),  conceptual  model  of  language  learning]. 

•  Results/Product.  A  short  description  of  what  was  accomplished/progress  toward 
meeting  the  objective. 

For  each  Phase  II  contract, 

•  If  the  Phase  I  did  not  become  a  Phase  II,  provide  a  short  reason  why  not  (e.g.,  No  - 
lack  of  funds,  No  -  approach  not  likely  to  succeed)  in  the  appropriate  cell. 

•  If  the  Phase  I  became  a  Phase  II,  indicate  in  the  appropriate  cell  why  it  was  selected 
for  a  Phase  II  (e.g.,  Yes  -  highly  creative  approach  to  training  language  skills). 
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SBIR/STTR  Phase  II  Contract 
Title 


Meaningful  graphic  depicting  the  Phase  II,  e.g., 
model,  software  flow  chart  or  product  design 


Meaningful  one-liner  about  what  the  graphic  represents  goes  here 


Company  name 


This  is  the  first  of  three  slides  (this  slide  and  the  two  following)  required  for  each 
Phase  II  contract  funded  under  the  SBIR/STTR  topic. 

•  Heading:  Indicate  the  Phase  II  contract  title. 

•  Use  the  slide  to  present  some  graphic  or  photographic  representation  of  the  contract 
effort  or  its  product. 

•  Place  a  caption  at  the  bottom  of  the  slide  that  highlights  the  graphic. 

•  Company.  Indicate  the  company  performing  the  SBIR/STTR  on  the  bottom  left  of 
the  slide. 
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This  is  the  second  slide  in  the  series  of  3  on  each  Phase  II  contract  within  a  topic. 

•  Heading:  Use  the  SBIR/STTR  contract  title. 

•  Objective:  A  short  bullet  on  the  objective  of  this  Phase  II.  (It  may  or  may  not  be  the 
same  as  the  Topic  objective  on  the  slide  one.) 

•  Technology  Solution:  What  technical  capability  or  “idea”  [e.g.,  latent  semantic 
analysis  (LSA),  machine-learning  techniques  to  extract  principles  from  stories, 
object-based  authoring,  SOAR  technology  (an  architecture  for  intelligent  problem 
solving  and  learning)  to  assess  cognitive  readiness,  phrenology]  was/will  be  applied 
to  achieve  the  objective  of  this  effort? 

•  Results/Findings:  In  no  more  than  three  bullets,  describe  the  results  and  their 
contribution  to  the  overall  objective  of  the  SBIR/STTR  topic.  Quantify  as  much  as 
possible.  If  the  Phase  II  is  still  in  effect,  provide  expected  contributions  that  Phase  II 
will  make  in  meeting  the  SBIR/STTR  topic  objective. 

•  Final  Deliverable:  Is  a  final  report  due?  If  so,  when.  What  form  will  this  final 
deliverable  take? 

•  Company.  Indicate  the  company  performing  the  SBIR/STTR  on  the  bottom  left  of 
the  slide. 
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SBIR/STTR  Phase  II  Contract 
Title 


Lessons  Learned 


Payoff 

Other 


6 


This  is  the  last  slide  in  the  series  of  three  on  each  contract  within  a  topic. 

•  Heading.  Use  the  SBIR/STTR  contract  title. 

•  Lessons  learned.  What  went/is  going  well?  What  did  not/is  not? 

•  Payoff 

-  A  bullet  on  the  return  on  investment  (general  knowledge  gained,  product 
implementation,  problem  solved,  and/or  impact  on  military  operational 
effectiveness). 

-  A  bullet  on  who  has  or  will  use  its  results  to  do  what?  (External  is  better,  but 
internal  is  all  right). 

-  For  those  not  yet  completed,  provide  potential  payoff. 

•  Other.  Anything  else  you  think  might  be  of  interest  to  the  audience  (e.g.,  an  award¬ 
winning  SBIR/STTR,  significant  publication,  new  capability  for  in-house  or  con¬ 
tractor  use,  generation  of  a  program  of  research  or  a  new  SBIR/STTR  topic,  move¬ 
ment  to  a  Phase  III). 

•  Company.  Indicate  the  company  performing  the  SBIR/STTR  on  the  bottom  left  of 
the  slide. 
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SBIR/STTR  Topic  Number  &  Title: 


Remaining  S&T  gaps: 


The  last  slide. 

•  Heading.  Use  the  SBIR/STTR  topic. 

•  Remaining  science  and  technology  (S&T)  gaps.  If  topic  is  completed,  use  no  more 

than  five  bullets  to  answer  the  most  appropriate  questions  posed  below: 

-  Does  the  problem  that  motivated  this  project  still  exist? 

-  What  S&T  gaps  remain  to  be  filled  in  this  topic?  What  remains  to  be  done  to 
solve  the  problem  or  resolve  the  issue  addressed  by  this  topic? 

-  Have  new  S&T  opportunities  emerged  or  are  there  additional  S&T  opportuni¬ 
ties  yet  to  be  examined? 

-  What  do  we  still  need  to  know  or  be  able  to  do? 

-  Has  research  on  this  topic  identified  a  new  problem/issue  that  should  be 
addressed  by  S&T  effort(s)?  If  yes,  please  suggest  some  S&T  gaps  that  need  to 
be  filled  to  address  this  new  problem/issue. 

If  the  Remaining  S&T  gaps  topic  is  not  completed,  provide  a  bullet  or  two  on  what  you 
would  do  next  on  this  topic. 
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